net: skb: isolate skb data area allocations into a separate bucket
Subsystems (6)
v6 cover net: skb: isolate skb data area allocations into a separate bucket
Hi!
This gets the buckets able to handle memcg (GFP_KERNEL_ACCOUNT) with isolation (since it's common due to AF_UNIX), and GFP_DMA with fall back (since it's rare). It gave me an excuse to build out bucket kunit tests too, and that (and LLM review) found a couple other issues that needed fixing too, including msg_msg allocations going uncharged to their memcg when CONFIG_SLAB_BUCKETS=n (fixed in 2/8).
Harry, on your v4 question[1] about bucket users giving their own
alignment: I tried that in v5, but a set's allocations don't always come from its own caches. With CONFIG_SLAB_BUCKETS=n, after a failed kmem_buckets_create(), and for the DMA and reclaimable fallbacks, they come from the general kmalloc caches, which can only give kmalloc()'s alignment. So v6 goes back to mirroring the kmalloc cache's alignment, and drops the ctor and flags arguments for the same reason. And the whole exploration made me realize I had a completely wrong understanding of how memcg worked. :P
The bulk of this is mm/slab, but the final patch is netdev, which Paolo acked in v4, so I'm hoping this whole series can go via slab?
Thanks!
-Kees
v6:
- drop v5's 6/7 ("Let a bucket set handle __GFP_ACCOUNT") and its
kmem_buckets_create_types(): memcg charges each object in whatever
cache serves it, so accounted allocations can stay in a set's single
row of caches, and the fallback now covers only DMA, reclaimable, and
no-obj-ext allocations (Sashiko)
- 2/8: new: account msg_msg with GFP_KERNEL_ACCOUNT again; with
CONFIG_SLAB_BUCKETS=n it went uncharged, since its accounting lived in
SLAB_ACCOUNT on bucket caches that are not created (Sashiko)
- 3/8: new: drop the ctor and flags arguments from kmem_buckets_create();
neither reaches allocations that fall back to the general kmalloc
caches, and no caller needs them any more (Sashiko)
- 4/8: go back to v4's form: no alignment argument, and each bucket cache
takes the alignment of the kmalloc cache it mirrors, since the fallbacks
to kmalloc can give no other (Sashiko, Harry)
- 5/8: say in the teardown comment that cache sharing comes from kmalloc
rounding sizes up to a larger class (Sashiko)
- 6/8: drop the explicit alignment tests; check that each size lands in
the cache of the size kmalloc() rounds it up to, not just a big enough
one; and in the destroy test, assert on the allocation, skip when KFENCE
serves it, and tear the set down through its KUnit cleanup action
(Sashiko)
- 7/8: keep __GFP_ACCOUNT allocations in the set, document what an
allocation that falls back loses, and test the reclaimable fallback
(Sashiko)
- 8/8: create the skb_data set with kmem_buckets_create(), and make the
comment above kmalloc_reserve() name no allocator (Sashiko)
- v5..v6 diff: https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/diff/?id=dev/v7.3-rc2/skb-buckets/v6&id2=dev/v7.3-rc2/skb-buckets/v5
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
[1] https://lore.kernel.org/all/arViR2Miz61-3fV4@thinkstation/
Kees Cook (7):
mm/slab: Mark the kmem_buckets_create() context as a Context: section ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again mm/slab: Drop the ctor and flags arguments from kmem_buckets_create() mm/slab: Give bucket caches the alignment of the caches they mirror mm/slab: Add kmem_buckets_destroy()
mm/slab: Add tests for the existing kmem_buckets behaviour mm/slab: Provide kmalloc type fallback for bucket allocations
Pedro Falcato (1):
net: skb: isolate skb data area allocations into a separate bucket
include/linux/slab.h | 6 +- mm/slab.h | 19 ++- ipc/msgutil.c | 8 +- lib/tests/slub_kunit.c | 291 +++++++++++++++++++++++++++++++++++++++++ mm/slab_common.c | 74 ++++++++--- mm/util.c | 2 +- net/core/skbuff.c | 10 +- 7 files changed, 378 insertions(+), 32 deletions(-)
Signature · 1 line
2.55.0
PATCH v6 1/8 mm/slab: Mark the kmem_buckets_create() context as a Context: section
kmem_buckets_create() had the same sentence about calling context as kmem_cache_create(), but lacked the "Context:" prefix, so kernel-doc rendered in the body instead of as a "context" section.
Give it the missing prefix. Additionally fix the "a interrupt" typo __kmem_cache_create_args() had.
--- mm/slab_common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/slab_common.c +2 -2
@@ -311,7 +311,7 @@ __kmem_cache_alias(const char *name, unsigned int size, slab_flags_t flags,* &SLAB_TYPESAFE_BY_RCU - Slab page (not individual objects) freeing delayed * by a grace period - see the full description before using. * -* Context: Cannot be called within a interrupt, but can be interrupted. +* Context: Cannot be called within an interrupt, but can be interrupted. * * Return: a pointer to the cache on success, NULL on failure. */
@@ -422,7 +422,7 @@ static struct kmem_cache *kmem_buckets_cache __ro_after_init;* to/from userspace. * @ctor: A constructor for the objects, run when new allocations are made. * -* Cannot be called within an interrupt, but can be interrupted. +* Context: Cannot be called within an interrupt, but can be interrupted. * * Return: a pointer to the cache on success, NULL on failure. When * CONFIG_SLAB_BUCKETS is not enabled, ZERO_SIZE_PTR is returned, and
Signature · 1 line
2.55.0
Re: [PATCH net-next v6 1/8] mm/slab: Mark the kmem_buckets_create() context as a Context: section
Quoted 7 lines · Kees Cook
kmem_buckets_create()had the same sentence about calling context askmem_cache_create(), but lacked the "Context:" prefix, so kernel-doc rendered in the body instead of as a "context" section.Give it the missing prefix. Additionally fix the "a interrupt" typo
__kmem_cache_create_args()had.
Quoted 6 lines
Fixes b32801d1255be ("mm/slab: Introduce kmem_buckets_create() and family")Assisted-by LLMReviewed-by Harry Yoo (Meta) <[email protected]>Acked-by Pedro Falcato <[email protected]>Reviewed-by Hao Li <[email protected]>Signed-off-by Kees Cook <[email protected]>
Quoted 26 lines
--- mm/slab_common.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)mm/slab_common.c +2
-2@@ -311,7 +311,7 @@ __kmem_cache_alias(const char *name, unsigned int size, slab_flags_t flags,* &SLAB_TYPESAFE_BY_RCU - Slab page (not individual objects) freeing delayed * by a grace period - see the full description before using. * -* Context: Cannot be called within a interrupt, but can be interrupted. +* Context: Cannot be called within an interrupt, but can be interrupted. * * Return: a pointer to the cache on success, NULL on failure. */ @@ -422,7 +422,7 @@ static struct kmem_cache *kmem_buckets_cache __ro_after_init;* to/from userspace. * @ctor: A constructor for the objects, run when new allocations are made. * -* Cannot be called within an interrupt, but can be interrupted. +* Context: Cannot be called within an interrupt, but can be interrupted. * * Return: a pointer to the cache on success, NULL on failure. When * CONFIG_SLAB_BUCKETS is not enabled, ZERO_SIZE_PTR is returned, and
PATCH v6 2/8 ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
In reply to Kees CookSince commit 734bbc1c97ea7 ("ipc, msg: Use dedicated slab buckets for alloc_msg()"), alloc_msg() allocates with GFP_KERNEL, and a msg_msg is accounted only through SLAB_ACCOUNT on its bucket caches. With CONFIG_SLAB_BUCKETS=n, kmem_buckets_create() creates no caches and kmem_buckets_alloc() is a GFP_KERNEL kmalloc() from the general caches, which do not account it; the same happens when kmem_buckets_create() fails. Either way, the allocation is not charged to the sender's memory cgroup.
Allocate with GFP_KERNEL_ACCOUNT again, as before that commit. Memcg charges such an allocation in whichever cache serves it, so drop the SLAB_ACCOUNT, which no longer adds anything.
Build tested ARCH=x86_64 defconfig with GCC 16.2.0, with CONFIG_SLAB_BUCKETS as y and n.
--- ipc/msgutil.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
ipc/msgutil.c +3 -2
@@ -43,7 +43,7 @@ static kmem_buckets *msg_buckets __ro_after_init;static int __init init_msg_buckets (void ) { - msg_buckets =kmem_buckets_create ("msg_msg" ,SLAB_ACCOUNT , + msg_buckets =kmem_buckets_create ("msg_msg" ,0 ,sizeof (struct msg_msg ),DATALEN_MSG ,NULL );
@@ -58,7 +58,8 @@ static struct msg_msg *alloc_msg(size_t len)size_t alen; alen =min (len,DATALEN_MSG ); - msg =kmem_buckets_alloc (msg_buckets,sizeof (*msg) + alen,GFP_KERNEL ); + msg =kmem_buckets_alloc (msg_buckets,sizeof (*msg) + alen, +GFP_KERNEL_ACCOUNT );if (msg ==NULL )return NULL ;
Signature · 1 line
2.55.0
Harry Yoo
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 10 lines · Kees Cook
Since commit
734bbc1c97ea7("ipc, msg: Use dedicated slab buckets foralloc_msg()"),alloc_msg()allocates withGFP_KERNEL, and amsg_msgis accounted only throughSLAB_ACCOUNTon its bucket caches.With
CONFIG_SLAB_BUCKETS=n,kmem_buckets_create()creates no caches andkmem_buckets_alloc()is aGFP_KERNELkmalloc()from the general caches, which do not account it; the same happens whenkmem_buckets_create()fails. Either way, the allocation is not charged to the sender's memory cgroup.
Ouch, now I see what's gone wrong here...
The fix for this bug should be Cc: stable IMHO.
Allowing to escape memcg charging is not good.
Quoted 3 lines
Allocate with
GFP_KERNEL_ACCOUNTagain, as before that commit. Memcg charges such an allocation in whichever cache serves it, so drop theSLAB_ACCOUNT, which no longer adds anything.
Hmm in the long term we don't want allowing __GFP_ACCOUNT allocations that are served from slab caches without SLAB_ACCOUNT, as this wastes memory.
See: https://lore.kernel.org/linux-mm/[email protected]
And now I see the initial kmem_buckets design did not sufficiently tackle the question "How this should work when kmem_buckets falls back to kmalloc?"
I suppose the kmem_buckets' abstraction should not be too tightly coupled with kmalloc caches. Creating a kmem_buckets should be conceptually equivalent to creating a set of caches with speicifc slab flags, size, align, useroffset/size. (for variable size allocation).
When it falls back to kmalloc, kmem_buckets itself should provide a compatibility layer when falling back to kmalloc.
(Okay, allowing ctor is completely broken, but other attributes are fine)
...I don't agree with the idea that "since kmem_buckets can fall back to kmalloc, kmem_buckets can only have the same requirements as kmalloc (slab flags, alignment, etc.)".
By that logic, shouldn't we give up specifying useroffset and usersize too? :-)
Quoted 33 lines
Build tested ARCH=x86_64 defconfig with GCC 16.2.0, with
CONFIG_SLAB_BUCKETSas y and n.Fixes 734bbc1c97ea7 ("ipc, msg: Use dedicated slab buckets for alloc_msg()")Assisted-by LLMSigned-off-by Kees Cook <[email protected]>--- ipc/msgutil.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)ipc/msgutil.c +3
-2@@ -43,7 +43,7 @@ static kmem_buckets *msg_buckets __ro_after_init;static int __init init_msg_buckets (void ) { - msg_buckets =kmem_buckets_create ("msg_msg" ,SLAB_ACCOUNT , + msg_buckets =kmem_buckets_create ("msg_msg" ,0 ,sizeof (struct msg_msg ),DATALEN_MSG ,NULL );@@ -58,7 +58,8 @@ static struct msg_msg *alloc_msg(size_t len)size_t alen; alen =min (len,DATALEN_MSG ); - msg =kmem_buckets_alloc (msg_buckets,sizeof (*msg) + alen,GFP_KERNEL ); + msg =kmem_buckets_alloc (msg_buckets,sizeof (*msg) + alen, +GFP_KERNEL_ACCOUNT );if (msg ==NULL )return NULL ;
Signature · 2 lines
Cheers, Harry / Hyeonggon
Harry Yoo
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 8 lines · Harry Yoo
And now I see the initial
kmem_bucketsdesign did not sufficiently tackle the question "How this should work whenkmem_bucketsfalls back to kmalloc?"I suppose the
kmem_buckets' abstraction should not be too tightly coupled with kmalloc caches. Creating akmem_bucketsshould be conceptually equivalent to creating a set of caches with speicifc slab flags, size, align, useroffset/size. (for variable size allocation).
Just to clarify... I have a few concerns on this area.
1. Complexity.
It's very counter-intuitive to understand how it works when SLAB_BUCKETS is not compiled. My brain cannot easily process two different implementations w/ and w/o SLAB_BUCKETS.
A "compatibility layer" allows us to assume certain expectations and ignore all implementation details when SLAB_BUCKETS is not compiled.
2. Making kmem_buckets tightly coupled with kmalloc's implementation details is not reasonable.
Let's say, at some point we make SLAB_BUCKETS always enabled and drop the config option. Then all kmem_buckets users will now have certain assumptions inherted from kmalloc (e.g. alignment), which is not reasonable.
3. __GFP_ACCOUNT and __GFP_DMA should be applied to all allocations in the slab cache. For kmalloc, these flags are only used to select the kmalloc cache (which has equivalent SLAB_* flags for __GFP_ACCOUNT or __GFP_DMA) to serve the allocation.
Specifying it on individual allocations (rather than as SLAB_ flags) makes it easy to break the assumption.
Quoted 2 lines
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.
For example, calculate_sizes() converts certain SLAB_* flags into GFP flags (s->allocflags) for allocation. We can have a similar logic when falling back to kmalloc.
Though current implementation does not allow this and requires some rework.
Signature · 2 lines
Cheers, Harry / Hyeonggon
Re: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 12 lines · Harry Yoo
Quoted 8 lines · Harry Yoo
And now I see the initial
kmem_bucketsdesign did not sufficiently tackle the question "How this should work whenkmem_bucketsfalls back to kmalloc?"I suppose the
kmem_buckets' abstraction should not be too tightly coupled with kmalloc caches. Creating akmem_bucketsshould be conceptually equivalent to creating a set of caches with speicifc slab flags, size, align, useroffset/size. (for variable size allocation).Just to clarify... I have a few concerns on this area.
Quoted 8 lines
1. Complexity.
It's very counter-intuitive to understand how it works when
SLAB_BUCKETSis not compiled. My brain cannot easily process two different implementations w/ and w/oSLAB_BUCKETS.A "compatibility layer" allows us to assume certain expectations and ignore all implementation details when
SLAB_BUCKETSis not compiled.
I'd rather make kmem_buckets not have a SLAB_BUCKETS option. :)
Quoted 7 lines
2. Making
kmem_bucketstightly coupled with kmalloc's implementation details is not reasonable.Let's say, at some point we make
SLAB_BUCKETSalways enabled and drop the config option. Then allkmem_bucketsusers will now have certain assumptions inherted from kmalloc (e.g. alignment), which is not reasonable.
Until it's always enabled, I don't see a reasonable way to separate those assumptions.
Quoted 7 lines
3.
__GFP_ACCOUNTand__GFP_DMAshould be applied to all allocations in the slab cache. For kmalloc, these flags are only used to select the kmalloc cache (which has equivalent SLAB_* flags for__GFP_ACCOUNTor__GFP_DMA) to serve the allocation.Specifying it on individual allocations (rather than as
SLAB_flags) makes it easy to break the assumption.
As in, you don't want a "pass-through" mode for __GFP_DMA?
It's a lot of overhead for __GFP_DMA to have multiple providers, so I'd rather allow kmem_buckets to allow pass-through (perhaps explicitly opt-in) where then the other options (e.g. alignment) must match the general kmalloc caches so that pass-through is consistent.
Quoted 9 lines
Quoted 2 lines
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.For example,
calculate_sizes()converts certain SLAB_* flags into GFP flags (s->allocflags) for allocation. We can have a similar logic when falling back to kmalloc.Though current implementation does not allow this and requires some rework.
I think the current series does everything correctly in the sense that nothing is allowing for mismatched fallbacks/passthrus. We could iterate from there once we have a need for this where a non-kmalloc-alignment is wanted, etc?
Signature · 1 line
Kees Cook
Harry Yoo
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 15 lines · Kees Cook
Quoted 14 lines · Harry Yoo
Quoted 8 lines · Harry Yoo
And now I see the initial
kmem_bucketsdesign did not sufficiently tackle the question "How this should work whenkmem_bucketsfalls back to kmalloc?"I suppose the
kmem_buckets' abstraction should not be too tightly coupled with kmalloc caches. Creating akmem_bucketsshould be conceptually equivalent to creating a set of caches with speicifc slab flags, size, align, useroffset/size. (for variable size allocation).Just to clarify... I have a few concerns on this area.
1. Complexity.
Quoted 8 lines
Quoted 6 lines
It's very counter-intuitive to understand how it works when
SLAB_BUCKETSis not compiled. My brain cannot easily process two different implementations w/ and w/oSLAB_BUCKETS.A "compatibility layer" allows us to assume certain expectations and ignore all implementation details when
SLAB_BUCKETSis not compiled.I'd rather make
kmem_bucketsnot have aSLAB_BUCKETSoption. :)
We'll need some data but looking at the discussion thread [1]:
Quoted 5 lines
I wasn't able to measure any performance impact at all. It does add a small bit of memory overhead, but it's on the order of a dozen pages used for the 2 extra sets of buckets. (E.g. it's well below the overhead introduced by
CONFIG_RANDOM_KMALLOC_CACHES, which adds 16 extra sets of buckets.)
[1] https://lore.kernel.org/all/202404280921.A7683D511@keescook
If the data still shows that the cost is still "dozens of pages" for a few buckets w/o measurable performance overhead, I think that's certainly doable.
Quoted 19 lines
Quoted 7 lines
2. Making
kmem_bucketstightly coupled with kmalloc's implementation details is not reasonable.Let's say, at some point we make
SLAB_BUCKETSalways enabled and drop the config option. Then allkmem_bucketsusers will now have certain assumptions inherted from kmalloc (e.g. alignment), which is not reasonable.Until it's always enabled, I don't see a reasonable way to separate those assumptions.
Quoted 7 lines
3.
__GFP_ACCOUNTand__GFP_DMAshould be applied to all allocations in the slab cache. For kmalloc, these flags are only used to select the kmalloc cache (which has equivalent SLAB_* flags for__GFP_ACCOUNTor__GFP_DMA) to serve the allocation.Specifying it on individual allocations (rather than as
SLAB_flags) makes it easy to break the assumption.
Quoted 6 lines
As in, you don't want a "pass-through" mode for
__GFP_DMA?It's a lot of overhead for
__GFP_DMAto have multiple providers, so I'd rather allowkmem_bucketsto allow pass-through (perhaps explicitly opt-in) where then the other options (e.g. alignment) must match the general kmalloc caches so that pass-through is consistent.
Having both "Specifying __GFP_DMA for slab caches without SLAB_DMA is illegal" and "However kmem_buckets is an exception because creating separate buckets for SLAB_DMA is costly" in mind is confusing and easy to get wrong...
Perhaps the user should bypass kmem_buckets and fall back to kmalloc?
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.For example,
calculate_sizes()converts certain SLAB_* flags into GFP flags (s->allocflags) for allocation. We can have a similar logic when falling back to kmalloc.
Quoted 7 lines
Quoted 2 lines
Though current implementation does not allow this and requires some rework.
I think the current series does everything correctly in the sense that nothing is allowing for mismatched fallbacks/passthrus. We could iterate from there once we have a need for this where a non-kmalloc-alignment is wanted, etc?
Making kmalloc tightly coupled with DMA alignment requirement was a mistake, and now it's too late to fix that. It's better to not repeat the same mistake again?
Signature · 2 lines
Cheers, Harry / Hyeonggon
Vlastimil Babka (SUSE)
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
In reply to Kees CookQuoted 15 lines · Kees Cook
Quoted 14 lines · Harry Yoo
Quoted 8 lines · Harry Yoo
And now I see the initial
kmem_bucketsdesign did not sufficiently tackle the question "How this should work whenkmem_bucketsfalls back to kmalloc?"I suppose the
kmem_buckets' abstraction should not be too tightly coupled with kmalloc caches. Creating akmem_bucketsshould be conceptually equivalent to creating a set of caches with speicifc slab flags, size, align, useroffset/size. (for variable size allocation).Just to clarify... I have a few concerns on this area.
1. Complexity.
Quoted 8 lines
Quoted 6 lines
It's very counter-intuitive to understand how it works when
SLAB_BUCKETSis not compiled. My brain cannot easily process two different implementations w/ and w/oSLAB_BUCKETS.A "compatibility layer" allows us to assume certain expectations and ignore all implementation details when
SLAB_BUCKETSis not compiled.I'd rather make
kmem_bucketsnot have aSLAB_BUCKETSoption. :)
The tradeoff hardening vs memory footprint exists however, so I don't think we can simply do that.
Quoted 10 lines
Quoted 7 lines
2. Making
kmem_bucketstightly coupled with kmalloc's implementation details is not reasonable.Let's say, at some point we make
SLAB_BUCKETSalways enabled and drop the config option. Then allkmem_bucketsusers will now have certain assumptions inherted from kmalloc (e.g. alignment), which is not reasonable.Until it's always enabled, I don't see a reasonable way to separate those assumptions.
Agree. I don't think the assumptions are so problematic.
Quoted 8 lines
Quoted 7 lines
3.
__GFP_ACCOUNTand__GFP_DMAshould be applied to all allocations in the slab cache. For kmalloc, these flags are only used to select the kmalloc cache (which has equivalent SLAB_* flags for__GFP_ACCOUNTor__GFP_DMA) to serve the allocation.Specifying it on individual allocations (rather than as
SLAB_flags) makes it easy to break the assumption.
Quoted 6 lines
As in, you don't want a "pass-through" mode for
__GFP_DMA?It's a lot of overhead for
__GFP_DMAto have multiple providers, so I'd rather allowkmem_bucketsto allow pass-through (perhaps explicitly opt-in) where then the other options (e.g. alignment) must match the general kmalloc caches so that pass-through is consistent.
"pass-through" mode for __GFP_DMA seems reasonable to me. I hope we eventually get rid of it, so I would support it in the most simple way possible.
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.For example,
calculate_sizes()converts certain SLAB_* flags into GFP flags (s->allocflags) for allocation. We can have a similar logic when falling back to kmalloc.
Quoted 7 lines
Quoted 2 lines
Though current implementation does not allow this and requires some rework.
I think the current series does everything correctly in the sense that nothing is allowing for mismatched fallbacks/passthrus. We could iterate from there once we have a need for this where a non-kmalloc-alignment is wanted, etc?
I think we can indeed simply make sure buckets have the same alignment as kmalloc. It comes naturally and should (more than) satisfy everyone. I doubt anyone would be needing a more specific alignment than that for a class of allocations with a variable size?
ctor removal is clear I think
for __GFP_ACCOUNT and __GFP_RECLAIMABLE I think what we could do is not to drop the flags argument kmem_buckets_create and instead require that it matches the usage:
__GFP_RECLAIMABLE requires SLAB_RECLAIM_ACCOUNT and vice versa and I can't imagine there would be a buckets instance that would allocate both kinds (either with or without __GFP_RECLAIMABLE)
SLAB_ACCOUNT requires __GFP_ACCOUNT, memcg objexts are always created and used
SLAB_MAY_ACCOUNT allows __GFP_ACCOUNT, memcg objexts are always created, and used when __GFP_ACCOUNT given
- perhaps not as optimal as the separate kmalloc types, but good enough?
__GFP_ACCOUNT without either SLAB_ACCOUNT or SLAB_MAY_ACCOUNT is wrong, __GFP_ACCOUNT would be ignored
with SLAB_BUCKETS=y these rules could be checked e.g. with a VM_WARN_ON_ONCE()
with SLAB_BUCKETS=n these rules won't be checked but also not matter in practice
wdyt?
Re: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 18 lines · Vlastimil Babka (SUSE)
Quoted 9 lines · Kees Cook
Quoted 8 lines · Harry Yoo
Quoted 2 lines · Harry Yoo
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.For example,
calculate_sizes()converts certain SLAB_* flags into GFP flags (s->allocflags) for allocation. We can have a similar logic when falling back to kmalloc.Quoted 7 lines
Quoted 2 lines
Though current implementation does not allow this and requires some rework.
I think the current series does everything correctly in the sense that nothing is allowing for mismatched fallbacks/passthrus. We could iterate from there once we have a need for this where a non-kmalloc-alignment is wanted, etc?
Quoted 4 lines
I think we can indeed simply make sure buckets have the same alignment as kmalloc. It comes naturally and should (more than) satisfy everyone. I doubt anyone would be needing a more specific alignment than that for a class of allocations with a variable size?
Right, the whole point of kmem_buckets was to replace general kmalloc uses, where no callers cared about alignment.
Quoted 1 line
ctor removal is clear I think
Completely. I was blinded by the API of kmem_cache_create(). :)
Quoted 7 lines
for
__GFP_ACCOUNTand__GFP_RECLAIMABLEI think what we could do is not to drop the flags argumentkmem_buckets_createand instead require that it matches the usage:
__GFP_RECLAIMABLErequiresSLAB_RECLAIM_ACCOUNTand vice versa and I can't imagine there would be a buckets instance that would allocate both kinds (either with or without__GFP_RECLAIMABLE)
Okay. (We have no users of this yet, FWIW.)
Quoted 1 line
SLAB_ACCOUNTrequires__GFP_ACCOUNT, memcg objexts are always created and used
Right.
Quoted 3 lines
SLAB_MAY_ACCOUNTallows__GFP_ACCOUNT, memcg objexts are always created, and used when__GFP_ACCOUNTgiven
- perhaps not as optimal as the separate kmalloc types, but good enough?
I think it's fine from studying what happens with the skb data. Though right now, SLAB_ACCOUNT and SLAB_MAY_ACCOUNT are functionally identical, yes?
Quoted 2 lines
__GFP_ACCOUNTwithout eitherSLAB_ACCOUNTorSLAB_MAY_ACCOUNTis wrong,__GFP_ACCOUNTwould be ignored
*ignored* is, I think, wrong here. It would create differing behavior between SLAB_BUCKETS=y/n. If we passed __GFP_ACCOUNT through to general kmalloc when the buckets lacked SLAB_ACCOUNT and SLAB_MAY_ACCOUNT, you'd lose the isolation, but not the accounting. (And could WARN_ONCE about it.) But if it was ignored, you'd lose accounting under SLAB_BUCKETS=y.
Quoted 3 lines
with SLAB_BUCKETS=y these rules could be checked e.g. with a
VM_WARN_ON_ONCE()with SLAB_BUCKETS=n these rules won't be checked but also not matter in practice
Agreed. That's what I was going for on this version. But like above, "SLAB_ACCOUNT requires __GFP_ACCOUNT" means what? Under SLAB_BUCKETS=y the allocation returns NULL? Or it passes through with WARN_ONCE? I'd rather not break something completely; better to fallback on a mismatch so the requested type happens (be it DMA, ACCOUNTing, etc).
Signature · 1 line
Kees Cook
Vlastimil Babka (SUSE)
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 39 lines · Kees Cook
Quoted 18 lines · Vlastimil Babka (SUSE)
Quoted 9 lines · Kees Cook
Quoted 8 lines · Harry Yoo
Quoted 2 lines · Harry Yoo
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.For example,
calculate_sizes()converts certain SLAB_* flags into GFP flags (s->allocflags) for allocation. We can have a similar logic when falling back to kmalloc.Quoted 7 lines
Quoted 2 lines
Though current implementation does not allow this and requires some rework.
I think the current series does everything correctly in the sense that nothing is allowing for mismatched fallbacks/passthrus. We could iterate from there once we have a need for this where a non-kmalloc-alignment is wanted, etc?
Quoted 4 lines
I think we can indeed simply make sure buckets have the same alignment as kmalloc. It comes naturally and should (more than) satisfy everyone. I doubt anyone would be needing a more specific alignment than that for a class of allocations with a variable size?
Right, the whole point of
kmem_bucketswas to replace general kmalloc uses, where no callers cared about alignment.Quoted 1 line
ctor removal is clear I think
Completely. I was blinded by the API of
kmem_cache_create(). :)Quoted 7 lines
for
__GFP_ACCOUNTand__GFP_RECLAIMABLEI think what we could do is not to drop the flags argumentkmem_buckets_createand instead require that it matches the usage:
__GFP_RECLAIMABLErequiresSLAB_RECLAIM_ACCOUNTand vice versa and I can't imagine there would be a buckets instance that would allocate both kinds (either with or without__GFP_RECLAIMABLE)
Quoted 9 lines
Okay. (We have no users of this yet, FWIW.)
Quoted 1 line
SLAB_ACCOUNTrequires__GFP_ACCOUNT, memcg objexts are always created and usedRight.
Quoted 3 lines
SLAB_MAY_ACCOUNTallows__GFP_ACCOUNT, memcg objexts are always created, and used when__GFP_ACCOUNTgiven
- perhaps not as optimal as the separate kmalloc types, but good enough?
Ah there's already a precedent, e.g. specifying __GFP_RECLAIM means you get a KMALLOC_RECLAIM cache, but it may or may not be accounted depending on passing also __GFP_ACCOUNT and there are no separate sets of slabs for these. Also same can happen with kmem_cache_create() cache when only some allocations are __GFP_ACCOUNT.
Quoted 3 lines
I think it's fine from studying what happens with the skb data. Though right now,
SLAB_ACCOUNTandSLAB_MAY_ACCOUNTare functionally identical, yes?
Both mean that objcg pointers might exist for the slabs from the cache, but SLAB_ACCOUNT also implies __GFP_ACCOUNT for every allocation from the cache even if not given. But for kmem_bucket_alloc() with SLAB_BUCKETS=n with we have nowhere to store that this implication should be done (that would be the "compatibility layer").
Quoted 5 lines
Quoted 2 lines
__GFP_ACCOUNTwithout eitherSLAB_ACCOUNTorSLAB_MAY_ACCOUNTis wrong,__GFP_ACCOUNTwould be ignored*ignored* is, I think, wrong here. It would create differing behavior between SLAB_BUCKETS=y/n.
Yes that's why we must avoid it. SLAB_MAY_ACCOUNT is a new thing and because there were already many caches that may or may not pass __GFP_ACCOUNT, we just assume it for now in __kmem_cache_create_args(). But that's a TODO to start tightening it. With buckets we only have a handful of users so we could try doing that right away, and have warnings in place to catch any changes that push it into the "wrong" state.
Quoted 5 lines
If we passed
__GFP_ACCOUNTthrough to general
kmalloc when the buckets lackedSLAB_ACCOUNTandSLAB_MAY_ACCOUNT, you'd lose the isolation, but not the accounting. (And couldWARN_ONCEabout it.) But if it was ignored, you'd lose accounting under SLAB_BUCKETS=y.
We could warn and fallback then, I guess.
Quoted 7 lines
Quoted 3 lines
with SLAB_BUCKETS=y these rules could be checked e.g. with a
VM_WARN_ON_ONCE()with SLAB_BUCKETS=n these rules won't be checked but also not matter in practice
Agreed. That's what I was going for on this version. But like above, "
SLAB_ACCOUNTrequires__GFP_ACCOUNT" means what? Under SLAB_BUCKETS=y the allocation returns NULL? Or it passes through withWARN_ONCE? I'd
"SLAB_ACCOUNT requires __GFP_ACCOUNT" means that if it's not done and SLAB_BUCKETS=n, then we get the issue that patch 2 fixes, and have no way to actually find out and warn that it's the case (with SLAB_BUCKETS=n... the warning can be issued with SLAB_BUCKETS=y and then it passes through as if there was __GFP_ACCOUNT).
Quoted 2 lines
rather not break something completely; better to fallback on a mismatch so the requested type happens (be it DMA, ACCOUNTing, etc).
Also instead of the scheme above, maybe we could explore the idea of "compatibility layer" too. It would mean that even with SLAB_BUCKETS=n, kmem_buckets_create() would create some object to hold the gfp_flags we want to append to all allocations from that bucket. But then kmem_buckets_alloc() would have to do something a bit more than map to __kmalloc_node_noprof (with PASS_KMALLOC_PARAMS directly). Of course SLAB_BUCKETS=y would be using this way too then to avoid further complication. Maybe the result would be more cleaner (simplifying PASS_KMALLOC_PARAMS and getting rid of PASS_BUCKET_PARAMS?) but I can't say immediately how well it would work.
Re: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
In reply to Harry YooQuoted 12 lines · Harry Yoo
Quoted 10 lines · Kees Cook
Since commit
734bbc1c97ea7("ipc, msg: Use dedicated slab buckets foralloc_msg()"),alloc_msg()allocates withGFP_KERNEL, and amsg_msgis accounted only throughSLAB_ACCOUNTon its bucket caches.With
CONFIG_SLAB_BUCKETS=n,kmem_buckets_create()creates no caches andkmem_buckets_alloc()is aGFP_KERNELkmalloc()from the general caches, which do not account it; the same happens whenkmem_buckets_create()fails. Either way, the allocation is not charged to the sender's memory cgroup.
Quoted 3 lines
Ouch, now I see what's gone wrong here...
The fix for this bug should be Cc: stable IMHO.
Allowing to escape memcg charging is not good.
Anything with Fixes: will end up in stable, so I just leave off the stable CC these days.
Quoted 9 lines
Quoted 3 lines
Allocate with
GFP_KERNEL_ACCOUNTagain, as before that commit. Memcg charges such an allocation in whichever cache serves it, so drop theSLAB_ACCOUNT, which no longer adds anything.Hmm in the long term we don't want allowing
__GFP_ACCOUNTallocations that are served from slab caches withoutSLAB_ACCOUNT, as this wastes memory.
This is how I thought things worked, so my original version of this series kept separate caches. (I can return to that too, but it doesn't seem to be needed today?)
Quoted 3 lines
And now I see the initial
kmem_bucketsdesign did not sufficiently tackle the question "How this should work whenkmem_bucketsfalls back to kmalloc?"
Right. We could just make kmem_buckets non-optional, too? Then they could have all the caller personalization they need.
Quoted 4 lines
I suppose the
kmem_buckets' abstraction should not be too tightly coupled with kmalloc caches. Creating akmem_bucketsshould be conceptually equivalent to creating a set of caches with speicifc slab flags, size, align, useroffset/size. (for variable size allocation).
In my mind, the general kmalloc caches are just a specific instance of a kmem_buckets...
Quoted 2 lines
When it falls back to kmalloc,
kmem_bucketsitself should provide a compatibility layer when falling back to kmalloc.
IMO, it'd just be better to make kmem_buckets not NEED a compat layer...
Quoted 8 lines
(Okay, allowing ctor is completely broken, but other attributes are fine)
...I don't agree with the idea that "since
kmem_bucketscan fall back to kmalloc,kmem_bucketscan only have the same requirements as kmalloc (slab flags, alignment, etc.)".By that logic, shouldn't we give up specifying useroffset and usersize too? :-)
Yup, but I kept that since only the hardening guarantees change under that condition -- nothing operationally depends on useroffset/size.
-Kees
Signature · 1 line
Kees Cook
Andrew Morton
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 2 lines · On Fri, 9 Oct 2026 20:14:20 -0700 Kees Cook <[email protected]> wrote:
Anything with Fixes: will end up in stable, so I just leave off the stable CC these days.
-stable maintainers have been asked to Not Do That for mm patches. Other subsystems are doing the same.
I don't know how they determine whether something is an mm patch. A patch which covers do-do-that and dont-do-that parts of the tree is likely to be mishandled.
So yeah, add that cc:stable to reduce the possibility of mistakes.
Vlastimil Babka (SUSE)
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 5 lines · Andrew Morton
Quoted 2 lines · On Fri, 9 Oct 2026 20:14:20 -0700 Kees Cook <[email protected]> wrote:
Anything with Fixes: will end up in stable, so I just leave off the stable CC these days.
Quoted 8 lines
-stable maintainers have been asked to Not Do That for mm patches. Other subsystems are doing the same.
I don't know how they determine whether something is an mm patch. A patch which covers do-do-that and dont-do-that parts of the tree is likely to be mishandled.
So yeah, add that cc:stable to reduce the possibility of mistakes.
There's also this from the list of pain points at
https://lwn.net/Articles/1099594/
Kroah-Hartman: subsystems that do not add CC: [email protected]
tags to patches, but which still expect the stable team to pick up the important fixes anyway. The stable maintainers are overwhelmed as it is and are not doing that.
Re: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 15 lines · Vlastimil Babka (SUSE)
Quoted 5 lines · Andrew Morton
Quoted 2 lines · On Fri, 9 Oct 2026 20:14:20 -0700 Kees Cook <[email protected]> wrote:
Anything with Fixes: will end up in stable, so I just leave off the stable CC these days.
Quoted 8 lines
-stable maintainers have been asked to Not Do That for mm patches. Other subsystems are doing the same.
I don't know how they determine whether something is an mm patch. A patch which covers do-do-that and dont-do-that parts of the tree is likely to be mishandled.
So yeah, add that cc:stable to reduce the possibility of mistakes.
Quoted 7 lines
There's also this from the list of pain points at
https://lwn.net/Articles/1099594/Kroah-Hartman: subsystems that do not add CC: [email protected]
tags to patches, but which still expect the stable team to pick up the important fixes anyway. The stable maintainers are overwhelmed as it is and are not doing that.
tl;dr: yup, sorry. I will add CC:stable in the future.
I went to go figure out where I had gotten this misunderstanding. netdev dropped their special handling in 2021 in commit dbbe7c962c3a ("docs: networking: drop special stable handling"). But I really thought I had Greg tell me (in person somewhere?) that "Cc: stable" was redundant, and I think what I misread in current docs was, "... such tagging is unnecessary if the stable team can derive the appropriate versions from Fixes: tags." in
https://docs.kernel.org/process/stable-kernel-rules.html#stable-kernel-rules
but that, on closer reading, is about the _versioning_ syntax of the "Cc: stable" trailer.
Signature · 1 line
Kees Cook
Vlastimil Babka (SUSE)
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
In reply to Kees CookQuoted 13 lines · Kees Cook
Since commit
734bbc1c97ea7("ipc, msg: Use dedicated slab buckets foralloc_msg()"),alloc_msg()allocates withGFP_KERNEL, and amsg_msgis accounted only throughSLAB_ACCOUNTon its bucket caches. WithCONFIG_SLAB_BUCKETS=n,kmem_buckets_create()creates no caches andkmem_buckets_alloc()is aGFP_KERNELkmalloc()from the general caches, which do not account it; the same happens whenkmem_buckets_create()fails. Either way, the allocation is not charged to the sender's memory cgroup.Allocate with
GFP_KERNEL_ACCOUNTagain, as before that commit. Memcg charges such an allocation in whichever cache serves it, so drop theSLAB_ACCOUNT, which no longer adds anything.
Quoted 6 lines
Build tested ARCH=x86_64 defconfig with GCC 16.2.0, with
CONFIG_SLAB_BUCKETSas y and n.Fixes 734bbc1c97ea7 ("ipc, msg: Use dedicated slab buckets for alloc_msg()")Assisted-by LLMSigned-off-by Kees Cook <[email protected]>
Could we, for the purposes of a backportable fix, just have both SLAB_ACCOUNT and GFP_KERNEL_ACCOUNT in place? That should do the right thing regardless of CONFIG_SLAB_BUCKETS state, no?
Quoted 27 lines
--- ipc/msgutil.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)ipc/msgutil.c +3
-2@@ -43,7 +43,7 @@ static kmem_buckets *msg_buckets __ro_after_init;static int __init init_msg_buckets (void ) { - msg_buckets =kmem_buckets_create ("msg_msg" ,SLAB_ACCOUNT , + msg_buckets =kmem_buckets_create ("msg_msg" ,0 ,sizeof (struct msg_msg ),DATALEN_MSG ,NULL );@@ -58,7 +58,8 @@ static struct msg_msg *alloc_msg(size_t len)size_t alen; alen =min (len,DATALEN_MSG ); - msg =kmem_buckets_alloc (msg_buckets,sizeof (*msg) + alen,GFP_KERNEL ); + msg =kmem_buckets_alloc (msg_buckets,sizeof (*msg) + alen, +GFP_KERNEL_ACCOUNT );if (msg ==NULL )return NULL ;
Vlastimil Babka (SUSE)
maintainerRe: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 17 lines · Vlastimil Babka (SUSE)
Quoted 16 lines · Kees Cook
Since commit
734bbc1c97ea7("ipc, msg: Use dedicated slab buckets foralloc_msg()"),alloc_msg()allocates withGFP_KERNEL, and amsg_msgis accounted only throughSLAB_ACCOUNTon its bucket caches. WithCONFIG_SLAB_BUCKETS=n,kmem_buckets_create()creates no caches andkmem_buckets_alloc()is aGFP_KERNELkmalloc()from the general caches, which do not account it; the same happens whenkmem_buckets_create()fails. Either way, the allocation is not charged to the sender's memory cgroup.Allocate with
GFP_KERNEL_ACCOUNTagain, as before that commit. Memcg charges such an allocation in whichever cache serves it, so drop theSLAB_ACCOUNT, which no longer adds anything.Build tested ARCH=x86_64 defconfig with GCC 16.2.0, with
CONFIG_SLAB_BUCKETSas y and n.
Quoted 7 lines
Quoted 3 lines
Fixes 734bbc1c97ea7 ("ipc, msg: Use dedicated slab buckets for alloc_msg()")Assisted-by LLMSigned-off-by Kees Cook <[email protected]>Could we, for the purposes of a backportable fix, just have both
SLAB_ACCOUNTandGFP_KERNEL_ACCOUNTin place? That should do the right thing regardless ofCONFIG_SLAB_BUCKETSstate, no?
BTW we're thinking to add 1+2 (once this part is clarified) for 7.4 (with stable@ for patch 2). But for the rest we'll have to target 7.5 as it's only a week to the 7.4 merge window now.
Re: [PATCH net-next v6 2/8] ipc, msg: Account msg_msg allocations with GFP_KERNEL_ACCOUNT again
Quoted 22 lines · Vlastimil Babka (SUSE)
Quoted 21 lines · Vlastimil Babka (SUSE)
Quoted 19 lines · Kees Cook
Since commit
734bbc1c97ea7("ipc, msg: Use dedicated slab buckets foralloc_msg()"),alloc_msg()allocates withGFP_KERNEL, and amsg_msgis accounted only throughSLAB_ACCOUNTon its bucket caches. WithCONFIG_SLAB_BUCKETS=n,kmem_buckets_create()creates no caches andkmem_buckets_alloc()is aGFP_KERNELkmalloc()from the general caches, which do not account it; the same happens whenkmem_buckets_create()fails. Either way, the allocation is not charged to the sender's memory cgroup.Allocate with
GFP_KERNEL_ACCOUNTagain, as before that commit. Memcg charges such an allocation in whichever cache serves it, so drop theSLAB_ACCOUNT, which no longer adds anything.Build tested ARCH=x86_64 defconfig with GCC 16.2.0, with
CONFIG_SLAB_BUCKETSas y and n.Fixes 734bbc1c97ea7 ("ipc, msg: Use dedicated slab buckets for alloc_msg()")Assisted-by LLMSigned-off-by Kees Cook <[email protected]>
Quoted 6 lines
Quoted 3 lines
Could we, for the purposes of a backportable fix, just have both
SLAB_ACCOUNTandGFP_KERNEL_ACCOUNTin place? That should do the right thing regardless ofCONFIG_SLAB_BUCKETSstate, no?BTW we're thinking to add 1+2 (once this part is clarified) for 7.4 (with stable@ for patch 2).
Yeah, I'll respin with it present on the create and the alloc sides.
Quoted 2 lines
But for the rest we'll have to target 7.5 as it's only a week to the 7.4 merge window now.
Yeah, that's totally fine; I'm in no rush. I want to get it right, and I'm still recovering from my surprise about how GFP_KERNEL_ACCOUNT actually works. ;)
Signature · 1 line
Kees Cook
PATCH v6 3/8 mm/slab: Drop the ctor and flags arguments from kmem_buckets_create()
In reply to Kees CookA bucket set passes its constructor and slab flags to its own caches, but its allocations do not always come from them. With CONFIG_SLAB_BUCKETS=n, kmem_buckets_create() returns ZERO_SIZE_PTR, and when creating the set fails it returns NULL. Either way kmem_buckets_alloc() is served by the general kmalloc caches, which have neither, so a caller cannot depend on them. msg_msg depended on SLAB_ACCOUNT, which it no longer passes since accounting through GFP_KERNEL_ACCOUNT instead.
No caller passes a constructor or flags; drop both arguments. The set's caches keep SLAB_NO_MERGE, which kmem_buckets_create() always added.
--- include/linux/slab.h | 5 ++--- ipc/msgutil.c | 5 ++--- mm/slab_common.c | 14 ++++---------- mm/util.c | 2 +- 4 files changed, 9 insertions(+), 17 deletions(-)
include/linux/slab.h +2 -3
@@ -890,9 +890,8 @@ void *kmem_cache_alloc_lru_noprof(struct kmem_cache *s, struct list_lru *lru,bool kmem_cache_charge (void *objp,gfp_t gfpflags);void kmem_cache_free (struct kmem_cache *s,void *objp); -kmem_buckets *kmem_buckets_create (const char *name,slab_flags_t flags, -unsigned int useroffset,unsigned int usersize, -void (*ctor)(void *)); +kmem_buckets *kmem_buckets_create (const char *name,unsigned int useroffset, +unsigned int usersize);/* * Bulk allocation and freeing operations. These are accelerated in an
ipc/msgutil.c +2 -3
@@ -43,9 +43,8 @@ static kmem_buckets *msg_buckets __ro_after_init;static int __init init_msg_buckets (void ) { - msg_buckets =kmem_buckets_create ("msg_msg" ,0 , -sizeof (struct msg_msg ), -DATALEN_MSG ,NULL ); + msg_buckets =kmem_buckets_create ("msg_msg" ,sizeof (struct msg_msg ), +DATALEN_MSG );return 0 ; }
mm/slab_common.c +4 -10
@@ -415,12 +415,10 @@ static struct kmem_cache *kmem_buckets_cache __ro_after_init;* allocations via kmem_buckets_alloc() * @name: A prefix string which is used in /proc/slabinfo to identify this * cache. The individual caches with have their sizes as the suffix. -* @flags: SLAB flags (see kmem_cache_create() for details). * @useroffset: Starting offset within an allocation that may be copied * to/from userspace. * @usersize: How many bytes, starting at @useroffset, may be copied * to/from userspace. -* @ctor: A constructor for the objects, run when new allocations are made. * * Context: Cannot be called within an interrupt, but can be interrupted. *
@@ -429,10 +427,8 @@ static struct kmem_cache *kmem_buckets_cache __ro_after_init;* subsequent calls to kmem_buckets_alloc() will fall back to kmalloc(). * (i.e. callers only need to check for NULL on failure.) */ -kmem_buckets *kmem_buckets_create (const char *name,slab_flags_t flags, -unsigned int useroffset, -unsigned int usersize, -void (*ctor)(void *)) +kmem_buckets *kmem_buckets_create (const char *name,unsigned int useroffset, +unsigned int usersize) {unsigned long mask =0 ;unsigned int idx;
@@ -455,8 +451,6 @@ kmem_buckets *kmem_buckets_create(const char *name, slab_flags_t flags,if (WARN_ON (!b))return NULL ; - flags |=SLAB_NO_MERGE ; -for (idx =0 ; idx <ARRAY_SIZE (kmalloc_caches[KMALLOC_NORMAL ]); idx++) {char *short_size, *cache_name;unsigned int cache_useroffset, cache_usersize;
@@ -487,8 +481,8 @@ kmem_buckets *kmem_buckets_create(const char *name, slab_flags_t flags,if (WARN_ON (!cache_name)) goto fail; (*b)[aligned_idx] =kmem_cache_create_usercopy (cache_name, size, -0 , flags, cache_useroffset, - cache_usersize, ctor); +0 ,SLAB_NO_MERGE , cache_useroffset, + cache_usersize,NULL );kfree (cache_name);if (WARN_ON (!(*b)[aligned_idx])) goto fail;
mm/util.c +1 -1
@@ -199,7 +199,7 @@ static kmem_buckets *user_buckets __ro_after_init;static int __init init_user_buckets (void ) { - user_buckets =kmem_buckets_create ("memdup_user" ,0 ,0 ,INT_MAX ,NULL ); + user_buckets =kmem_buckets_create ("memdup_user" ,0 ,INT_MAX );return 0 ; }
Signature · 1 line
2.55.0
PATCH v6 4/8 mm/slab: Give bucket caches the alignment of the caches they mirror
In reply to Kees CookA bucket set is created with kmem_cache_create_usercopy(..., align = 0), so calculate_alignment() falls back to arch_slab_minalign(), typically 8 bytes. The general kmalloc caches it stands in for are created through create_boot_cache(), which starts from ARCH_KMALLOC_MINALIGN and raises it to the largest power-of-two divisor of the size:
if (flags & SLAB_KMALLOC) align = max(align, 1U << (ffs(size) - 1));
This is only a problem when slab metadata is enabled with CONFIG_KASAN=y, CONFIG_SLUB_DEBUG_ON=y, or "slab_debug=...", because metadata changes the stride size off a power of two, for example:
size 128: bucket align=8 size=224 | kmalloc align=128 size=384 size 512: bucket align=8 size=608 | kmalloc align=512 size=1536 size 2048: bucket align=8 size=2144 | kmalloc align=2048 size=6144
So bucket allocations will fail the IS_ALIGNED(p, ARCH_DMA_MINALIGN) check, potentially creating problems for non-coherent DMA situation.
Take the alignment from the kmalloc cache being mirrored, which is where the size and the name suffix already come from, so a caller moving from kmalloc() keeps the alignment it had.
Don't take an alignment from the caller instead. With CONFIG_SLAB_BUCKETS=n, or when kmem_buckets_create() fails, kmem_buckets_alloc() is served by the general kmalloc caches, which give kmalloc()'s alignment and no other.
--- mm/slab_common.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/slab_common.c +2 -1
@@ -481,7 +481,8 @@ kmem_buckets *kmem_buckets_create(const char *name, unsigned int useroffset,if (WARN_ON (!cache_name)) goto fail; (*b)[aligned_idx] =kmem_cache_create_usercopy (cache_name, size, -0 ,SLAB_NO_MERGE , cache_useroffset, + kmalloc_caches[KMALLOC_NORMAL ][idx]->align, +SLAB_NO_MERGE , cache_useroffset, cache_usersize,NULL );kfree (cache_name);if (WARN_ON (!(*b)[aligned_idx]))
Signature · 1 line
2.55.0
PATCH v6 5/8 mm/slab: Add kmem_buckets_destroy()
In reply to Kees Cookkmem_buckets_create() intentionally had no "destroy" counterpart. Every caller has lived in core kernel code and creates its set once at boot, so nothing has needed to take one down. However, KUnit tests may be built module, so we need it now to support the coming tests.
Some caches have size aliases, so the same pointer is stored at more then one index, so we have to save it, clear all matching instances, and then free the saved cache pointer. (This is what the bitmap was tracking before in the "allocation failed" error path.)
When CONFIG_SLAB_BUCKETS=n the whole body compiles away, matching the ZERO_SIZE_PTR that kmem_buckets_create() hands back in that configuration.
--- include/linux/slab.h | 1 + mm/slab_common.c | 50 +++++++++++++++++++++++++++++++++++++------- 2 files changed, 44 insertions(+), 7 deletions(-)
include/linux/slab.h +1 -0
@@ -892,6 +892,7 @@ void kmem_cache_free(struct kmem_cache *s, void *objp);kmem_buckets *kmem_buckets_create (const char *name,unsigned int useroffset,unsigned int usersize); +void kmem_buckets_destroy (kmem_buckets *bucket);/* * Bulk allocation and freeing operations. These are accelerated in an
mm/slab_common.c +43 -7
76 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v6 6/8 mm/slab: Add tests for the existing kmem_buckets behaviour
In reply to Kees Cookkmem_buckets has had no test coverage since it was added. Add tests, including stuff unique to the bucket design:
- A bucket allocation comes from a cache of the set's own, and that cache carries SLAB_NO_MERGE.
- Each size is served by a cache of the set, of the size kmalloc() rounds it up to, including 96 and 192, which are not powers of two and which kmalloc shares with a larger class on some configurations. Sizes above KMALLOC_MAX_CACHE_SIZE go to the page allocator instead, bucket set or not.
- Each cache is aligned like the kmalloc cache it mirrors.
- With CONFIG_SLAB_BUCKETS=n, kmem_buckets_create() still returns a non-NULL (zero size alloc pointer), so that callers only have to check for failure, and allocations through it come from the general caches.
- Destroying a set takes its caches down rather than only freeing the set, which is what a module creating one on each load depends on. A set freed without its caches would leave the names taken, and the next load would warn about every one of them. This test skips when KFENCE serves its allocation, since a KFENCE object is in none of the cache's slabs for the teardown to find.
The tests skip rather than compile out, which is useful for testing the CONFIG_SLAB_BUCKETS=n behaviors.
Built and tests passing (with expected skips) on ARCH=x86_64 defconfig with GCC 16.2.0, with CONFIG_SLAB_BUCKETS as y and n.
--- lib/tests/slub_kunit.c | 228 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 228 insertions(+)
lib/tests/slub_kunit.c +228 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #include <kunit/test.h> #include <kunit/test-bug.h> +#include <kunit/resource.h> #include <linux/mm.h> #include <linux/slab.h> #include <linux/module.h>
@@ -9,6 +10,7 @@#include <linux/delay.h> #include <linux/perf_event.h> #include <linux/kprobes.h> +#include <linux/kfence.h> #include "../mm/slab.h" static struct kunit_resource resource;
@@ -474,6 +476,227 @@ static int test_init(struct kunit *test)return 0 ; } +/* Destroy buckets on test exit so a failed KUNIT_ASSERT_*() doesn't leak. */ +KUNIT_DEFINE_ACTION_WRAPPER(destroy_buckets, kmem_buckets_destroy, kmem_buckets *); + +#define KUNIT_ASSERT_BUCKETS_CREATED (test, b) \ + do { \ + KUNIT_ASSERT_NOT_NULL(test, b); \ + KUNIT_ASSERT_EQ(test, 0, \ + kunit_add_action_or_reset(test, \ + destroy_buckets, b)); \ + } while (0) + +/* +* The cache an allocation came from, or NULL if it came from no cache at +* all, e.g. a size too big for any of them is served by the page allocator. +*/ +static struct kmem_cache *cache_of (void *p) +{ +struct slab *slab =virt_to_slab (p); + +return slab ? slab->slab_cache :NULL ; +} + +/* +* A bucket set exists to keep its allocations out of the caches everything +* else uses, so check the two things that make that true: they come from a +* cache of the set's own, and that cache is never merged into another. +*/ +static void test_kmem_buckets_isolation (struct kunit *test) +{ +struct kmem_cache *bucket_cache, *general_cache; +kmem_buckets *b; +void *p, *q; + +if (!IS_ENABLED (CONFIG_SLAB_BUCKETS )) +kunit_skip (test,"needs CONFIG_SLAB_BUCKETS" ); + + b =kmem_buckets_create ("isolated_buckets" ,0 ,INT_MAX ); +KUNIT_ASSERT_BUCKETS_CREATED (test, b); + +/* +* Free each allocation before asserting on the next one: the cache +* outlives its objects, so nothing below needs them, and an assertion +* that leaves one behind would make the deferred teardown report a +* cache that is still in use. +*/ + p =kmem_buckets_alloc (b,128 ,GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + bucket_cache =cache_of (p); +kfree (p); +KUNIT_ASSERT_NOT_NULL (test, bucket_cache); + +KUNIT_EXPECT_TRUE_MSG (test,strstarts (bucket_cache->name,"isolated_buckets-" ), +"expected a bucket cache, got %s" , bucket_cache->name); + +/* +* Cache merging is on by default, and a bucket cache merged into a +* same-sized general one would quietly undo the whole separation. +*/ +KUNIT_EXPECT_TRUE (test, bucket_cache->flags &SLAB_NO_MERGE ); + + q =kmalloc (128 ,GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, q); + general_cache =cache_of (q); +kfree (q); +KUNIT_ASSERT_NOT_NULL (test, general_cache); + +KUNIT_EXPECT_PTR_NE (test, bucket_cache, general_cache); +} + +/* +* Each size is served by a cache of the set, of the size kmalloc() rounds it +* up to, including the size classes that are not powers of two, which kmalloc +* shares with a larger class on some configurations. Sizes past the largest +* cache are served by the page allocator, bucket set or not. +*/ +static void test_kmem_buckets_sizes (struct kunit *test) +{ +static const size_t sizes[] = {8 ,96 ,192 ,1024 ,4096 }; +struct kmem_cache *c; +kmem_buckets *b; +void *p; +int i; + +if (!IS_ENABLED (CONFIG_SLAB_BUCKETS )) +kunit_skip (test,"needs CONFIG_SLAB_BUCKETS" ); + + b =kmem_buckets_create ("sized_buckets" ,0 ,INT_MAX ); +KUNIT_ASSERT_BUCKETS_CREATED (test, b); + +for (i =0 ; i <ARRAY_SIZE (sizes); i++) { + p =kmem_buckets_alloc (b, sizes[i],GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + c =cache_of (p); +kfree (p); +KUNIT_ASSERT_NOT_NULL (test, c); + +KUNIT_EXPECT_TRUE_MSG (test,strstarts (c->name,"sized_buckets-" ), +"size %zu: expected a bucket cache, got %s" , + sizes[i], c->name); +KUNIT_EXPECT_EQ_MSG (test, c->object_size, +kmalloc_size_roundup (sizes[i]), +"size %zu: served by %s" , sizes[i], c->name); + } + +/* Too big for any cache: a folio from the page allocator, not a slab. */ + p =kmem_buckets_alloc (b,KMALLOC_MAX_CACHE_SIZE +1 ,GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + c =cache_of (p); +kfree (p); + +KUNIT_EXPECT_NULL (test, c); +} + +/* +* A bucket cache stands in for a kmalloc cache, so it has to be aligned like +* one. The DMA layer decides whether a buffer needs bouncing from its size, +* on the grounds that a kmalloc cache of that size is already aligned for +* the device, so a weaker alignment here is not something a caller can see +* coming. Without slab debugging the size implies the alignment and this +* holds either way; with it, only the cache's own alignment does. +*/ +static void test_kmem_buckets_alignment (struct kunit *test) +{ +static const size_t sizes[] = {128 ,512 ,2048 }; +struct kmem_cache *bucket_cache, *general_cache; +kmem_buckets *b; +void *p; +int i; + +if (!IS_ENABLED (CONFIG_SLAB_BUCKETS )) +kunit_skip (test,"needs CONFIG_SLAB_BUCKETS" ); + + b =kmem_buckets_create ("aligned_buckets" ,0 ,INT_MAX ); +KUNIT_ASSERT_BUCKETS_CREATED (test, b); + +for (i =0 ; i <ARRAY_SIZE (sizes); i++) { + p =kmem_buckets_alloc (b, sizes[i],GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + bucket_cache =cache_of (p); +KUNIT_EXPECT_TRUE_MSG (test, +IS_ALIGNED ((unsigned long )p,ARCH_DMA_MINALIGN ), +"size %zu: object %p is not %d byte aligned" , + sizes[i], p, (int )ARCH_DMA_MINALIGN ); +kfree (p); + + p =kmalloc (sizes[i],GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + general_cache =cache_of (p); +kfree (p); + +KUNIT_ASSERT_NOT_NULL (test, bucket_cache); +KUNIT_ASSERT_NOT_NULL (test, general_cache); +KUNIT_EXPECT_EQ_MSG (test, bucket_cache->align, general_cache->align, +"size %zu: bucket cache aligned to %u, %s to %u" , + sizes[i], bucket_cache->align, + general_cache->name, general_cache->align); + } +} + +/* +* With the feature compiled out, kmem_buckets_create() still returns +* something non-NULL so that callers only have to check for failure, and +* allocations through it work (i.e. come from the general caches). +*/ +static void test_kmem_buckets_disabled (struct kunit *test) +{ +kmem_buckets *b; +struct kmem_cache *c; +void *p; + +if (IS_ENABLED (CONFIG_SLAB_BUCKETS )) +kunit_skip (test,"only meaningful without CONFIG_SLAB_BUCKETS" ); + + b =kmem_buckets_create ("disabled_buckets" ,0 ,INT_MAX ); +KUNIT_ASSERT_BUCKETS_CREATED (test, b); + + p =kmem_buckets_alloc (b,128 ,GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + c =cache_of (p); +kfree (p); +KUNIT_ASSERT_NOT_NULL (test, c); + +KUNIT_EXPECT_TRUE_MSG (test, !strstarts (c->name,"disabled_buckets-" ), +"expected a general cache, got %s" , c->name); +} + +/* Destroying a set has to take its caches down, not just free the set. */ +static void test_kmem_buckets_destroy (struct kunit *test) +{ +kmem_buckets *b; +void *p; + +if (!IS_ENABLED (CONFIG_SLAB_BUCKETS )) +kunit_skip (test,"needs CONFIG_SLAB_BUCKETS" ); + + b =kmem_buckets_create ("destroyed_buckets" ,0 ,INT_MAX ); +KUNIT_ASSERT_BUCKETS_CREATED (test, b); + +/* +* Deliberately leaked, as test_leak_destroy() leaks its own: the +* teardown below has to find it. kmem_cache_destroy() unlists the +* cache either way, so the name is still released. +*/ + p =kmem_buckets_alloc (b,128 ,GFP_KERNEL ); +KUNIT_ASSERT_NOT_NULL (test, p); + +/* +* A KFENCE object is in none of the cache's slabs, so the teardown +* would not find it to report. +*/ +if (is_kfence_address (p)) { +kfree (p); +kunit_skip (test,"the allocation came from KFENCE" ); + } + +/* Tear the set down now, rather than at exit, to check the report. */ +kunit_release_action (test, destroy_buckets, b); + +KUNIT_EXPECT_EQ (test,2 , slab_errors); +} +static struct kunit_case test_cases[] = {KUNIT_CASE (test_clobber_zone),
@@ -495,6 +718,11 @@ static struct kunit_case test_cases[] = {#if defined(CONFIG_KPROBES ) && defined(CONFIG_SMP )KUNIT_CASE_SLOW (test_kmalloc_nolock_and_friends_kprobe), #endif +KUNIT_CASE (test_kmem_buckets_isolation), +KUNIT_CASE (test_kmem_buckets_sizes), +KUNIT_CASE (test_kmem_buckets_alignment), +KUNIT_CASE (test_kmem_buckets_disabled), + KUNIT_CASE(test_kmem_buckets_destroy), {} };
Signature · 1 line
2.55.0
PATCH v6 7/8 mm/slab: Provide kmalloc type fallback for bucket allocations
In reply to Kees Cookkmem_buckets_create() clones kmalloc_caches[KMALLOC_NORMAL]. kmalloc_slab() figures out the kmalloc type the caller asks for, but then ignored it whenever a bucket set was in use, returning a normal cache regardless. That breaks an allocation that needs other pages: a GFP_DMA allocation would not get memory from ZONE_DMA, and a __GFP_RECLAIMABLE one would miss the reclaimable caches. None of the current users do this, so there is no problem, but it makes adding new users fragile. For example, skb data[1] needs to handle GFP_DMA (rarely).
Send those allocations to the general caches instead, so nothing breaks and regular allocations remain isolated in the set. Accounted allocations stay in the set: memcg charges each object on its own, in any cache, so a bucket cache serves them as well as kmalloc-cg-* does.
Built and tests pass with ARCH=x86_64 defconfig with GCC 16.2.0, with CONFIG_SLAB_BUCKETS as y and n, and with CONFIG_MEMCG as y, n, and y with "cgroup.memory=nokmem".
--- mm/slab.h | 19 +++++++++++-- lib/tests/slub_kunit.c | 63 ++++++++++++++++++++++++++++++++++++++++++ mm/slab_common.c | 5 ++++ 3 files changed, 85 insertions(+), 2 deletions(-)
mm/slab.h +17 -2
@@ -421,6 +421,22 @@ static inline unsigned int size_index_elem(unsigned int bytes)return (bytes -1 ) /8 ; } +/* +* Which set of buckets to use for the given kmalloc_cache_type. A bucket set +* mirrors the KMALLOC_NORMAL caches, and also serves accounted allocations: +* memcg charges each object on its own, in any cache. Types that need +* different pages (DMA, reclaimable) or no obj_exts fall back to the +* general caches. +*/ +static inline kmem_buckets * +kmalloc_choose_bucket (kmem_buckets *bucket,enum kmalloc_cache_type type) +{ +if (bucket && (type <=KMALLOC_PARTITION_END || type ==KMALLOC_CGROUP )) +return bucket; + +return &kmalloc_caches[type]; +} +/* * Find the kmem_cache structure that serves a given size of * allocation
@@ -438,8 +454,7 @@ kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, kmalloc_token_t token,if (alloc_flags &SLAB_ALLOC_NO_OBJ_EXT ) type =KMALLOC_NO_OBJ_EXT ; -if (!b) - b = &kmalloc_caches[type]; + b =kmalloc_choose_bucket (b, type);if (size <=192 ) index = kmalloc_size_index[size_index_elem (size)];else
lib/tests/slub_kunit.c +63 -0
75 lines not on this page · raw message
mm/slab_common.c +5 -0
@@ -420,6 +420,11 @@ static struct kmem_cache *kmem_buckets_cache __ro_after_init;* @usersize: How many bytes, starting at @useroffset, may be copied * to/from userspace. * +* Accounted (__GFP_ACCOUNT) allocations are served by the set like any +* other. Allocations that need DMA or reclaimable memory are served by the +* general kmalloc caches instead, without the set's isolation or usercopy +* region. +* * Context: Cannot be called within an interrupt, but can be interrupted. * * Return: a pointer to the cache on success, NULL on failure. When
Signature · 1 line
2.55.0
PATCH v6 8/8 net: skb: isolate skb data area allocations into a separate bucket
In reply to Kees CookFrom: Pedro Falcato <[email protected]>
SKB data area allocations (as done from alloc_skb()) use kmalloc(). These allocations can be variably sized and their contents can be more or less controlled from userspace, which makes them useful for attackers that want to overwrite a use-after-free'd object from the same kmalloc slab (which often just requires the sizes to roughly match into the same kmalloc bucket). [0] is an easy example of an exploit that uses netlink skb allocation to target another similarly-sized accidentally freed object.
While other mitigations like CONFIG_RANDOM_KMALLOC_CACHES exist, these are probabilistic. Use the existing kmem buckets API to further isolate these allocations in a guaranteed fashion, when CONFIG_SLAB_BUCKETS=y.
AF_UNIX sets sk_allocation to GFP_KERNEL_ACCOUNT, and those skb data areas, the ones most worth isolating, stay in the set, where memcg charges them as it would in the general caches. GFP_DMA falls back to the general caches, being passed to an skb allocator only by rare devices.
--- net/core/skbuff.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
net/core/skbuff.c +7 -3
@@ -586,6 +586,8 @@ struct sk_buff *napi_build_skb(void *data, unsigned int frag_size) }EXPORT_SYMBOL (napi_build_skb); +static kmem_buckets *skb_data_buckets__ro_after_init ; +static void *kmalloc_pfmemalloc (size_t obj_size,gfp_t flags,int node) {if (!gfp_pfmemalloc_allowed (flags))
@@ -593,11 +595,12 @@ static void *kmalloc_pfmemalloc(size_t obj_size, gfp_t flags, int node)if (!obj_size)return kmem_cache_alloc_node (net_hotdata.skb_small_head_cache, flags, node); -return kmalloc_node_track_caller (obj_size, flags, node); +return kmem_buckets_alloc_node_track_caller (skb_data_buckets, obj_size, + flags, node); }/* -* kmalloc_reserve is a wrapper around kmalloc_node_track_caller that tells +* kmalloc_reserve is a wrapper around a caller-tracked kmalloc that tells * the caller if emergency pfmemalloc reserves are being used. If it is and * the socket is later found to be SOCK_MEMALLOC then PFMEMALLOC reserves * may be used. Otherwise, the packet data may be discarded until enough
@@ -634,7 +637,7 @@ static void *kmalloc_reserve(unsigned int *size, gfp_t flags, int node,* Try a regular allocation, when that fails and we're not entitled * to the reserves, fail. */ - obj =kmalloc_node_track_caller (obj_size, + obj =kmem_buckets_alloc_node_track_caller (skb_data_buckets, obj_size, flags | __GFP_NOMEMALLOC | __GFP_NOWARN, node);if (likely (obj))
@@ -5235,6 +5238,7 @@ void __init skb_init(void)0 ,SKB_SMALL_HEAD_HEADROOM , NULL); + skb_data_buckets =kmem_buckets_create ("skb_data" ,0 ,INT_MAX );skb_extensions_init (); }
Signature · 1 line
2.55.0
Harry Yoo
maintainerQuoted 1 line · Kees Cook
Hi!
Hi Kees!
Was hoping to say hi to you at LPC but I missed the chance ;) Maybe next time. Safe travels!
Uh, my mailbox stopped working for a few days as I forgot to renew subscription. (Kiryl told me it's bouncing, thanks!) Hopefully I didn't miss too much...
Quoted 2 lines
This gets the buckets able to handle memcg (
GFP_KERNEL_ACCOUNT) with isolation (since it's common due toAF_UNIX),
Cool!
Quoted 7 lines
and
GFP_DMAwith fall back
(since it's rare).It gave me an excuse to build out bucket kunit tests
too, and that (and LLM review) found a couple other issues that needed fixing too, includingmsg_msgallocations going uncharged to their memcg whenCONFIG_SLAB_BUCKETS=n(fixed in 2/8).
Oh.
Quoted 8 lines
Harry, on your v4 question[1] about bucket users giving their own
alignment: I tried that in v5, but a set's allocations don't always come from its own caches. WithCONFIG_SLAB_BUCKETS=n, after a failedkmem_buckets_create(), and for the DMA and reclaimable fallbacks, they come from the general kmalloc caches, which can only give kmalloc()'s alignment.So v6 goes back to mirroring the kmalloc cache's alignment,
I might be missing something, but why is that a problem?
For kmem_buckets users, the reason* to specify alignment is because they might need less strict alignment than kmalloc.
(*Perhaps it's nice to document that in the comment)
However, because kmem_buckets can fall back to kmalloc on e.g. kernels w/o CONFIG_SLAB_BUCKETS, it should be fine to fall back. No?
Creating kmem_buckets with more strict alignment than kmalloc doesn't make sense.
Quoted 1 line
and drops the ctor and flags arguments for the same reason.
Uh, for ctor and flags, yes. We can't have them in kmem_buckets.
Quoted 6 lines
And the whole
exploration made me realize I had a completely wrong understanding of how memcg worked. :PThe bulk of this is mm/slab, but the final patch is netdev, which Paolo acked in v4, so I'm hoping this whole series can go via slab?
Going thorough slab/for-next sounds reasonable to me once it gets some reviews.
Vlastimil?
Signature · 2 lines
Cheers, Harry / Hyeonggon
Quoted 7 lines · Harry Yoo
Quoted 1 line · Kees Cook
Hi!
Hi Kees!
Was hoping to say hi to you at LPC but I missed the chance ;) Maybe next time. Safe travels!
Hi! Yes, I kept trying to find you and Vlastimil but it never worked out. LPC is a non-stop hallway track usually. :) I will try again next year!
Quoted 12 lines
Quoted 1 line
So v6 goes back to mirroring the kmalloc cache's alignment,
I might be missing something, but why is that a problem?
For
kmem_bucketsusers, the reason* to specify alignment is because they might need less strict alignment than kmalloc.(*Perhaps it's nice to document that in the comment)
However, because
kmem_bucketscan fall back to kmalloc on e.g. kernels w/oCONFIG_SLAB_BUCKETS, it should be fine to fall back. No?
Quoted 6 lines
Creating
kmem_bucketswith more strict alignment than kmalloc doesn't make sense.Quoted 1 line
and drops the ctor and flags arguments for the same reason.
Uh, for ctor and flags, yes. We can't have them in
kmem_buckets.
Yeah, and given that these two, I'd just prefer to keep it a direct mirror for alignment too and not allow for any configurability here:
they are supposed to be direct stand-ins for the general cache.
Quoted 2 lines
Going thorough slab/for-next sounds reasonable to me once it gets some reviews.
Thanks!
-Kees
Signature · 1 line
Kees Cook
Quoted 57 lines · Harry Yoo
Quoted 1 line · Kees Cook
Hi!
Hi Kees!
Was hoping to say hi to you at LPC but I missed the chance ;) Maybe next time. Safe travels!
Uh, my mailbox stopped working for a few days as I forgot to renew subscription. (Kiryl told me it's bouncing, thanks!) Hopefully I didn't miss too much...
Quoted 2 lines
This gets the buckets able to handle memcg (
GFP_KERNEL_ACCOUNT) with isolation (since it's common due toAF_UNIX),Cool!
Quoted 7 lines
and
GFP_DMAwith fall back
(since it's rare).It gave me an excuse to build out bucket kunit tests
too, and that (and LLM review) found a couple other issues that needed fixing too, includingmsg_msgallocations going uncharged to their memcg whenCONFIG_SLAB_BUCKETS=n(fixed in 2/8).Oh.
Quoted 8 lines
Harry, on your v4 question[1] about bucket users giving their own
alignment: I tried that in v5, but a set's allocations don't always come from its own caches. WithCONFIG_SLAB_BUCKETS=n, after a failedkmem_buckets_create(), and for the DMA and reclaimable fallbacks, they come from the general kmalloc caches, which can only give kmalloc()'s alignment.So v6 goes back to mirroring the kmalloc cache's alignment,
I might be missing something, but why is that a problem?
For
kmem_bucketsusers, the reason* to specify alignment is because they might need less strict alignment than kmalloc.(*Perhaps it's nice to document that in the comment)
However, because
kmem_bucketscan fall back to kmalloc on e.g. kernels w/oCONFIG_SLAB_BUCKETS, it should be fine to fall back. No?Creating
kmem_bucketswith more strict alignment than kmalloc doesn't make sense.Quoted 1 line
and drops the ctor and flags arguments for the same reason.
Uh, for ctor and flags, yes. We can't have them in
kmem_buckets.Quoted 4 lines
And the whole
exploration made me realize I had a completely wrong understanding of how memcg worked. :P
Quoted 7 lines
Quoted 2 lines
The bulk of this is mm/slab, but the final patch is netdev, which Paolo acked in v4, so I'm hoping this whole series can go via slab?
Going thorough slab/for-next sounds reasonable to me once it gets some reviews.
Vlastimil?
Sure! If you review and feel it's ready to be added there, please do so. I couldn't yet due to conferencing, should be able next week.