slab: Introduce dedicated bucket allocator

Introduce CONFIG_SLAB_BUCKETS which provides the infrastructure to support separated kmalloc buckets (in the following kmem_buckets_create() patches and future codetag-based separation). Since this will provide a mitigation for a very common case of exploits, enable it by default.

To be able to choose which buckets to allocate from, make the buckets available to the internal kmalloc interfaces by adding them as the first argument, rather than depending on the buckets being chosen from

the fixed set of global buckets. Where the bucket is not available, pass NULL, which means "use the default system kmalloc bucket set" (the prior existing behavior), as implemented in kmalloc_slab().

To avoid adding the extra argument when !CONFIG_SLAB_BUCKETS, only the top-level macros and static inlines use the buckets argument (where they are stripped out and compiled out respectively). The actual extern functions can then been built without the argument, and the internals fall back to the global kmalloc buckets unconditionally.

mm/Kconfig +5 -0

@@ -273,6 +273,22 @@ config SLAB_FREELIST_HARDENED
 	  sacrifices to harden the kernel slab allocator against common
 	  freelist exploit methods.
 
+config SLAB_BUCKETS
+	bool "Support allocation from separate kmalloc buckets"
+	depends on !SLUB_TINY
+	help
+	  Kernel heap attacks frequently depend on being able to create

On 6/19/24 9:33 PM, Kees Cook wrote:
I was wondering why I don't see the buckets in slabinfo and turns out it was SLAB_MERGE_DEFAULT. It would probably make sense for SLAB_MERGE_DEFAULT to depends on !SLAB_BUCKETS now as the merging defeats the purpose, wdyt?

Dedicated caches are available for fixed size allocations via kmem_cache_alloc(), but for dynamically sized allocations there is only the global kmalloc API's set of buckets available. This means it isn't possible to separate specific sets of dynamically sized allocations into a separate collection of caches.

This leads to a use-after-free exploitation weakness in the Linux kernel since many heap memory spraying/grooming attacks depend on using userspace-controllable dynamically sized allocations to collide with fixed size allocations that end up in same cache.

While CONFIG_RANDOM_KMALLOC_CACHES provides a probabilistic defense against these kinds of "type confusion" attacks, including for fixed same-size heap objects, we can create a complementary deterministic defense for dynamically sized allocations that are directly user controlled. Addressing these cases is limited in scope, so isolating these kinds of interfaces will not become an unbounded game of whack-a-mole. For example, many pass through memdup_user(), making isolation there very effective.

In order to isolate user-controllable dynamically-sized allocations from the common system kmalloc allocations, introduce kmem_buckets_create(), which behaves like kmem_cache_create(). Introduce kmem_buckets_alloc(), which behaves like kmem_cache_alloc(). Introduce kmem_buckets_alloc_track_caller() for where caller tracking is needed. Introduce kmem_buckets_valloc() for cases where vmalloc fallback is needed.

This can also be used in the future to extend allocation profiling's use of code tagging to implement per-caller allocation cache isolation[1] even for dynamic allocations.

Memory allocation pinning[2] is still needed to plug the Use-After-Free cross-allocator weakness, but that is an existing and separate issue which is complementary to this improvement. Development continues for that feature via the SLAB_VIRTUAL[3] series (which could also provide guard pages -- another complementary improvement).

Link https://lore.kernel.org/lkml/202402211449.401382D2AF@keescook [1] open here

Link https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html [2]

Link https://lore.kernel.org/lkml/[email protected]/ [3] open here

Signed-off-by Kees Cook <[email protected]>

---
 include/linux/slab.h | 13 ++++++++
 mm/slab_common.c     | 78 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)

include/linux/slab.h +0 -0

index 8d0800c7579a..3698b15b6138 100644

Quoted 4 lines · Kees Cook

Introduce CONFIG_SLAB_BUCKETS which provides the infrastructure to support separated kmalloc buckets (in the following kmem_buckets_create() patches and future codetag-based separation). Since this will provide a mitigation for a very common case of exploits, enable it by default.

No longer "enable it by default".

Quoted 5 lines

To be able to choose which buckets to allocate from, make the buckets available to the internal kmalloc interfaces by adding them as the first argument, rather than depending on the buckets being chosen from

second argument now

Quoted 5 lines

the fixed set of global buckets. Where the bucket is not available, pass NULL, which means "use the default system kmalloc bucket set" (the prior existing behavior), as implemented in kmalloc_slab().

Quoted 5 lines

To avoid adding the extra argument when !CONFIG_SLAB_BUCKETS, only the top-level macros and static inlines use the buckets argument (where they are stripped out and compiled out respectively). The actual extern functions can then been built without the argument, and the internals fall back to the global kmalloc buckets unconditionally.

Also describes the previous implementation and not the new one?

Quoted 17 lines

mm/Kconfig +11 -0

@@ -273,6 +273,22 @@ config SLAB_FREELIST_HARDENED
 	  sacrifices to harden the kernel slab allocator against common
 	  freelist exploit methods.
 
+config SLAB_BUCKETS
+	bool "Support allocation from separate kmalloc buckets"
+	depends on !SLUB_TINY
+	help
+	  Kernel heap attacks frequently depend on being able to create
+	  specifically-sized allocations with user-controlled contents
+	  that will be allocated into the same kmalloc bucket as a
+	  target object. To avoid sharing these allocation buckets,
+	  provide an explicitly separated set of buckets to be used for
+	  user-controlled allocations. This may very slightly increase
+	  memory fragmentation, though in practice it's only a handful

mm/Kconfig config SLAB_FREELIST_HARDENED · line 290

Quoted 4 lines

mm/Kconfig +4 -0

+	  of extra pages since the bulk of user-controlled allocations
+	  are relatively long-lived.
+
+	  If unsure, say Y.

I was wondering why I don't see the buckets in slabinfo and turns out it was SLAB_MERGE_DEFAULT. It would probably make sense for SLAB_MERGE_DEFAULT to depends on !SLAB_BUCKETS now as the merging defeats the purpose, wdyt?

Quoted 4 lines · Vlastimil Babka

On 6/19/24 9:33 PM, Kees Cook wrote:
I was wondering why I don't see the buckets in slabinfo and turns out it was SLAB_MERGE_DEFAULT. It would probably make sense for SLAB_MERGE_DEFAULT to depends on !SLAB_BUCKETS now as the merging defeats the purpose, wdyt?

Hm I might have been just blind, can see them there now. Anyway it probably doesn't make much sense to have SLAB_BUCKETS and/or RANDOM_KMALLOC_CACHES together with SLAB_MERGE_DEFAULT?

Quoted 55 lines · Kees Cook

Dedicated caches are available for fixed size allocations via kmem_cache_alloc(), but for dynamically sized allocations there is only the global kmalloc API's set of buckets available. This means it isn't possible to separate specific sets of dynamically sized allocations into a separate collection of caches.

This leads to a use-after-free exploitation weakness in the Linux kernel since many heap memory spraying/grooming attacks depend on using userspace-controllable dynamically sized allocations to collide with fixed size allocations that end up in same cache.

While CONFIG_RANDOM_KMALLOC_CACHES provides a probabilistic defense against these kinds of "type confusion" attacks, including for fixed same-size heap objects, we can create a complementary deterministic defense for dynamically sized allocations that are directly user controlled. Addressing these cases is limited in scope, so isolating these kinds of interfaces will not become an unbounded game of whack-a-mole. For example, many pass through memdup_user(), making isolation there very effective.

In order to isolate user-controllable dynamically-sized allocations from the common system kmalloc allocations, introduce kmem_buckets_create(), which behaves like kmem_cache_create(). Introduce kmem_buckets_alloc(), which behaves like kmem_cache_alloc(). Introduce kmem_buckets_alloc_track_caller() for where caller tracking is needed. Introduce kmem_buckets_valloc() for cases where vmalloc fallback is needed.

This can also be used in the future to extend allocation profiling's use of code tagging to implement per-caller allocation cache isolation[1] even for dynamic allocations.

Memory allocation pinning[2] is still needed to plug the Use-After-Free cross-allocator weakness, but that is an existing and separate issue which is complementary to this improvement. Development continues for that feature via the SLAB_VIRTUAL[3] series (which could also provide guard pages -- another complementary improvement).

Link https://lore.kernel.org/lkml/202402211449.401382D2AF@keescook [1] open here

Link https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html [2]

Link https://lore.kernel.org/lkml/[email protected]/ [3] open here

Signed-off-by Kees Cook <[email protected]>

---
 include/linux/slab.h | 13 ++++++++
 mm/slab_common.c     | 78 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)

include/linux/slab.h +0 -0

index 8d0800c7579a..3698b15b6138 100644
@@ -549,6 +549,11 @@ void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru,
 
 void kmem_cache_free(struct kmem_cache *s, void *objp);
 

include/linux/slab.h void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru, · line 555

Quoted 4 lines

include/linux/slab.h +4 -0

+kmem_buckets *kmem_buckets_create(const char *name, unsigned int align,
+				  slab_flags_t flags,
+				  unsigned int useroffset, unsigned int usersize,
+				  void (*ctor)(void *));

I'd drop the ctor, I can't imagine how it would be used with variable-sized allocations.

Probably also "align" doesn't make much sense since we're just copying the kmalloc cache sizes and its implicit alignment of any power-of-two allocations.

I don't think any current kmalloc user would
suddenly need either of those as you convert it to buckets, and definitely not any user converted automatically by the code tagging.

Quoted 11 lines · Vlastimil Babka

include/linux/slab.h void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru, · line 555

Quoted 8 lines

include/linux/slab.h +4 -0

@@ -549,6 +549,11 @@ void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru,
 
 void kmem_cache_free(struct kmem_cache *s, void *objp);
 
+kmem_buckets *kmem_buckets_create(const char *name, unsigned int align,
+				  slab_flags_t flags,
+				  unsigned int useroffset, unsigned int usersize,
+				  void (*ctor)(void *));

I'd drop the ctor, I can't imagine how it would be used with variable-sized allocations.

I've kept it because for "kmalloc wrapper" APIs, e.g. devm_kmalloc(), there is some "housekeeping" that gets done explicitly right now that I think would be better served by using a ctor in the future. These APIs are variable-sized, but have a fixed size header, so they have a "minimum size" that the ctor can still operate on, etc.

Quoted 3 lines

Probably also "align" doesn't make much sense since we're just copying the kmalloc cache sizes and its implicit alignment of any power-of-two allocations.

Yeah, that's probably true. I kept it since I wanted to mirror kmem_cache_create() to make this API more familiar looking.

Quoted 3 lines

I don't think any current kmalloc user would
suddenly need either of those as you convert it to buckets, and definitely not any user converted automatically by the code tagging.

Right, it's not needed for either the explicit users nor the future automatic users. But since these arguments are available internally, there seems to be future utility, it's not fast path, and it made things feel like the existing API, I'd kind of like to keep it.

But all that said, if you really don't want it, then sure I can drop those arguments. Adding them back in the future shouldn't be too much churn.

Quoted 9 lines · Vlastimil Babka

Quoted 4 lines · Vlastimil Babka

On 6/19/24 9:33 PM, Kees Cook wrote:
I was wondering why I don't see the buckets in slabinfo and turns out it was SLAB_MERGE_DEFAULT. It would probably make sense for SLAB_MERGE_DEFAULT to depends on !SLAB_BUCKETS now as the merging defeats the purpose, wdyt?

Hm I might have been just blind, can see them there now. Anyway it probably doesn't make much sense to have SLAB_BUCKETS and/or RANDOM_KMALLOC_CACHES together with SLAB_MERGE_DEFAULT?

It's already handled so that the _other_ caches can still be merged if people want it. See new_kmalloc_cache():

#ifdef CONFIG_RANDOM_KMALLOC_CACHES
        if (type >= KMALLOC_RANDOM_START && type <= KMALLOC_RANDOM_END)
                flags |= SLAB_NO_MERGE;

#endif

Dedicated caches are available for fixed size allocations via kmem_cache_alloc(), but for dynamically sized allocations there is only the global kmalloc API's set of buckets available. This means it isn't possible to separate specific sets of dynamically sized allocations into a separate collection of caches.

This leads to a use-after-free exploitation weakness in the Linux kernel since many heap memory spraying/grooming attacks depend on using userspace-controllable dynamically sized allocations to collide with fixed size allocations that end up in same cache.

While CONFIG_RANDOM_KMALLOC_CACHES provides a probabilistic defense against these kinds of "type confusion" attacks, including for fixed same-size heap objects, we can create a complementary deterministic defense for dynamically sized allocations that are directly user controlled. Addressing these cases is limited in scope, so isolating these kinds of interfaces will not become an unbounded game of whack-a-mole. For example, many pass through memdup_user(), making isolation there very effective.

Quoted 11 lines · Kees Cook

Dedicated caches are available for fixed size allocations via kmem_cache_alloc(), but for dynamically sized allocations there is only the global kmalloc API's set of buckets available. This means it isn't possible to separate specific sets of dynamically sized allocations into a separate collection of caches.

This leads to a use-after-free exploitation weakness in the Linux kernel since many heap memory spraying/grooming attacks depend on using userspace-controllable dynamically sized allocations to collide with fixed size allocations that end up in same cache.

Quoted 8 lines

While CONFIG_RANDOM_KMALLOC_CACHES provides a probabilistic defense against these kinds of "type confusion" attacks, including for fixed same-size heap objects, we can create a complementary deterministic defense for dynamically sized allocations that are directly user controlled. Addressing these cases is limited in scope, so isolating these kinds of interfaces will not become an unbounded game of whack-a-mole. For example, many pass through memdup_user(), making isolation there very effective.

Isn't the attack still possible if the attacker can free the slab page during the use-after-free period with enough memory pressure?

Someone else might grab the page that was in the bucket for another slab and the type confusion could hurt again.

Or is there some other defense against that, other than CONFIG_DEBUG_PAGEALLOC or full slab poisoning? And how expensive does it get when any of those are enabled?

I remember reading some paper about a apple allocator trying similar techniques and it tried very hard to never reuse memory (probably not a good idea for Linux though)

I assume you thought about this, but it would be good to discuss such limitations and interactions in the commit log.

Quoted 8 lines

Memory allocation pinning[2] is still needed to plug the Use-After-Free cross-allocator weakness, but that is an existing and separate issue which is complementary to this improvement. Development continues for that feature via the SLAB_VIRTUAL[3] series (which could also provide guard pages -- another complementary improvement).

[...]

Link https://googleprojectzero.blogspot.com/2021/10/how-simple-linux-kernel-memory.html [2]

Link https://lore.kernel.org/lkml/[email protected]/ [3] open here

functions can then been built without the argument, …

+++ b/include/linux/slab.h

-void *kmalloc_node_track_caller_noprof(size_t size, gfp_t flags, int node,

+void *__kmalloc_node_track_caller_noprof(DECL_BUCKET_PARAMS(size, b), gfp_t flags, int node,

Quoted 19 lines · Kees Cook

Quoted 11 lines · Vlastimil Babka

include/linux/slab.h void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru, · line 555

Quoted 8 lines

include/linux/slab.h +4 -0

@@ -549,6 +549,11 @@ void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru,
 
 void kmem_cache_free(struct kmem_cache *s, void *objp);
 
+kmem_buckets *kmem_buckets_create(const char *name, unsigned int align,
+				  slab_flags_t flags,
+				  unsigned int useroffset, unsigned int usersize,
+				  void (*ctor)(void *));

I'd drop the ctor, I can't imagine how it would be used with variable-sized allocations.

I've kept it because for "kmalloc wrapper" APIs, e.g. devm_kmalloc(), there is some "housekeeping" that gets done explicitly right now that I think would be better served by using a ctor in the future. These APIs are variable-sized, but have a fixed size header, so they have a "minimum size" that the ctor can still operate on, etc.

Quoted 6 lines

Quoted 3 lines

Probably also "align" doesn't make much sense since we're just copying the kmalloc cache sizes and its implicit alignment of any power-of-two allocations.

Yeah, that's probably true. I kept it since I wanted to mirror kmem_cache_create() to make this API more familiar looking.

Rust people were asking about kmalloc alignment (but I forgot the details)

so maybe this could be useful for them? CC rust-for-linux.

Quoted 4 lines

Quoted 3 lines

I don't think any current kmalloc user would
suddenly need either of those as you convert it to buckets, and definitely not any user converted automatically by the code tagging.

Quoted 8 lines

Right, it's not needed for either the explicit users nor the future automatic users. But since these arguments are available internally, there seems to be future utility, it's not fast path, and it made things feel like the existing API, I'd kind of like to keep it.

But all that said, if you really don't want it, then sure I can drop those arguments. Adding them back in the future shouldn't be too much churn.

I guess we can keep it then.

Quoted 27 lines · Vlastimil Babka

Quoted 25 lines · Kees Cook

Quoted 11 lines · Vlastimil Babka

include/linux/slab.h void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru, · line 555

Quoted 8 lines

include/linux/slab.h +4 -0

@@ -549,6 +549,11 @@ void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru,
 
 void kmem_cache_free(struct kmem_cache *s, void *objp);
 
+kmem_buckets *kmem_buckets_create(const char *name, unsigned int align,
+				  slab_flags_t flags,
+				  unsigned int useroffset, unsigned int usersize,
+				  void (*ctor)(void *));

I'd drop the ctor, I can't imagine how it would be used with variable-sized allocations.

I've kept it because for "kmalloc wrapper" APIs, e.g. devm_kmalloc(), there is some "housekeeping" that gets done explicitly right now that I think would be better served by using a ctor in the future. These APIs are variable-sized, but have a fixed size header, so they have a "minimum size" that the ctor can still operate on, etc.

Quoted 3 lines

Probably also "align" doesn't make much sense since we're just copying the kmalloc cache sizes and its implicit alignment of any power-of-two allocations.

Yeah, that's probably true. I kept it since I wanted to mirror kmem_cache_create() to make this API more familiar looking.

Quoted 1 line

Rust people were asking about kmalloc alignment (but I forgot the details)

It was me! The ask is whether we can specify the alignment for the allocation API, for example, requesting a size=96 and align=32 memory, or the allocation API could do a "best alignment", for example, allocating a size=96 will give a align=32 memory. As far as I understand, kmalloc() doesn't support this.

Quoted 2 lines

so maybe this could be useful for them? CC rust-for-linux.

I took a quick look as what kmem_buckets is, and seems to me that align doesn't make sense here (and probably not useful in Rust as well) because a kmem_buckets is a set of kmem_caches, each has its own object size, making them share the same alignment is probably not what you want. But I could be missing something.

Regards,
Boqun

Quoted 15 lines

Quoted 4 lines

Quoted 3 lines

I don't think any current kmalloc user would
suddenly need either of those as you convert it to buckets, and definitely not any user converted automatically by the code tagging.

Quoted 8 lines

Right, it's not needed for either the explicit users nor the future automatic users. But since these arguments are available internally, there seems to be future utility, it's not fast path, and it made things feel like the existing API, I'd kind of like to keep it.

But all that said, if you really don't want it, then sure I can drop those arguments. Adding them back in the future shouldn't be too much churn.

I guess we can keep it then.

Quoted 28 lines · Boqun Feng

Quoted 27 lines · Vlastimil Babka

Quoted 25 lines · Kees Cook

Quoted 11 lines · Vlastimil Babka

include/linux/slab.h void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru, · line 555

Quoted 8 lines

include/linux/slab.h +4 -0

@@ -549,6 +549,11 @@ void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru,
 
 void kmem_cache_free(struct kmem_cache *s, void *objp);
 
+kmem_buckets *kmem_buckets_create(const char *name, unsigned int align,
+                                 slab_flags_t flags,
+                                 unsigned int useroffset, unsigned int usersize,
+                                 void (*ctor)(void *));

I'd drop the ctor, I can't imagine how it would be used with variable-sized allocations.

I've kept it because for "kmalloc wrapper" APIs, e.g. devm_kmalloc(), there is some "housekeeping" that gets done explicitly right now that I think would be better served by using a ctor in the future. These APIs are variable-sized, but have a fixed size header, so they have a "minimum size" that the ctor can still operate on, etc.

Quoted 3 lines

Probably also "align" doesn't make much sense since we're just copying the kmalloc cache sizes and its implicit alignment of any power-of-two allocations.

Yeah, that's probably true. I kept it since I wanted to mirror kmem_cache_create() to make this API more familiar looking.

Quoted 7 lines

Quoted 1 line

Rust people were asking about kmalloc alignment (but I forgot the details)

It was me! The ask is whether we can specify the alignment for the allocation API, for example, requesting a size=96 and align=32 memory, or the allocation API could do a "best alignment", for example, allocating a size=96 will give a align=32 memory. As far as I understand, kmalloc() doesn't support this.

Hm yeah we only have guarantees for power-or-2 allocations.

Quoted 8 lines

Quoted 2 lines

so maybe this could be useful for them? CC rust-for-linux.

I took a quick look as what kmem_buckets is, and seems to me that align doesn't make sense here (and probably not useful in Rust as well) because a kmem_buckets is a set of kmem_caches, each has its own object size, making them share the same alignment is probably not what you want. But I could be missing something.

How flexible do you need those alignments to be? Besides the power-of-two guarantees, we currently have only two odd sizes with 96 and 192. If those were guaranteed to be aligned 32 bytes, would that be sufficient? Also do you ever allocate anything smaller than 32 bytes then?

To summarize, if Rust's requirements can be summarized by some rules and it's not completely ad-hoc per-allocation alignment requirement (or if it is, does it have an upper bound?) we could perhaps figure out the creation of rust-specific kmem_buckets to give it what's needed?

Quoted 13 lines

Quoted 7 lines

Quoted 1 line

I took a quick look as what kmem_buckets is, and seems to me that align doesn't make sense here (and probably not useful in Rust as well) because a kmem_buckets is a set of kmem_caches, each has its own object size, making them share the same alignment is probably not what you want. But I could be missing something.

How flexible do you need those alignments to be? Besides the power-of-two guarantees, we currently have only two odd sizes with 96 and 192. If those were guaranteed to be aligned 32 bytes, would that be sufficient? Also do you ever allocate anything smaller than 32 bytes then?

Quoted 4 lines

To summarize, if Rust's requirements can be summarized by some rules and it's not completely ad-hoc per-allocation alignment requirement (or if it is, does it have an upper bound?) we could perhaps figure out the creation of rust-specific kmem_buckets to give it what's needed?

Rust's allocator API can take any size and alignment as long as:

1. The alignment is a power of two.
2. The size is non-zero.
3. When you round up the size to the next multiple of the alignment, then it must not overflow the signed type isize / ssize_t.

What happens right now is that when Rust wants an allocation with a higher alignment than ARCH_SLAB_MINALIGN, then it will increase size until it becomes a power of two so that the power-of-two guarantee gives a properly aligned allocation.

Alice

Quoted 25 lines · Alice Ryhl

Quoted 13 lines

Quoted 7 lines

Quoted 1 line

I took a quick look as what kmem_buckets is, and seems to me that align doesn't make sense here (and probably not useful in Rust as well) because a kmem_buckets is a set of kmem_caches, each has its own object size, making them share the same alignment is probably not what you want. But I could be missing something.

How flexible do you need those alignments to be? Besides the power-of-two guarantees, we currently have only two odd sizes with 96 and 192. If those were guaranteed to be aligned 32 bytes, would that be sufficient? Also do you ever allocate anything smaller than 32 bytes then?

Quoted 4 lines

To summarize, if Rust's requirements can be summarized by some rules and it's not completely ad-hoc per-allocation alignment requirement (or if it is, does it have an upper bound?) we could perhaps figure out the creation of rust-specific kmem_buckets to give it what's needed?

Rust's allocator API can take any size and alignment as long as:

1. The alignment is a power of two.
2. The size is non-zero.
3. When you round up the size to the next multiple of the alignment, then it must not overflow the signed type isize / ssize_t.

Quoted 4 lines

What happens right now is that when Rust wants an allocation with a higher alignment than ARCH_SLAB_MINALIGN, then it will increase size until it becomes a power of two so that the power-of-two guarantee gives a properly aligned allocation.

So am I correct thinking that, if the cache of size 96 bytes guaranteed a 32byte alignment, and 192 bytes guaranteed 64byte alignment, and the rest of sizes with the already guaranteed power-of-two alignment, then on rust side you would only have to round up sizes to the next multiples of the alignemnt (rule 3 above) and that would be sufficient?
Abstracting from the specific sizes of 96 and 192, the guarantee on kmalloc
side would have to be - guarantee alignment to the largest power-of-two divisor of the size. Does that sound right?

Then I think we could have some flag for kmem_buckets creation that would do the right thing.

Quoted 1 line · Vlastimil Babka

Rust people were asking about kmalloc alignment (but I forgot the details)

It was me! The ask is whether we can specify the alignment for the allocation API, for example, requesting a size=96 and align=32 memory, or the allocation API could do a "best alignment", for example, allocating a size=96 will give a align=32 memory. As far as I understand, kmalloc() doesn't support this.

Quoted 3 lines · Boqun Feng

Quoted 1 line · Vlastimil Babka

Rust people were asking about kmalloc alignment (but I forgot the details)

Quoted 5 lines

It was me! The ask is whether we can specify the alignment for the allocation API, for example, requesting a size=96 and align=32 memory, or the allocation API could do a "best alignment", for example, allocating a size=96 will give a align=32 memory. As far as I understand, kmalloc() doesn't support this.

I can drop the "align" argument. Do we want to hard-code a per-cache-size alignment for the caches in a kmem_buckets collection?