Skip to content

mm/slab, alloc_tag: reduce obj_ext memory waste

Vlastimil Babka (SUSE) cgroups linux-mm lkml 3 versions – In review

Get series with b4
b4 am [email protected] change-id 20260714-b4-objext_split-da82426257d5

Versions newest first

  1. v3 Latest13 patches 25 repliesIn review
  2. v213 patches 32 repliesSuperseded
  3. v1 RFC12 patches 30 repliesSuperseded

Changes in v3 from the cover letter

- Review tags from Harry, Hao, Suren - thanks!
- Patch "skip handle_failed_objexts_alloc() with profiling disabled"
removed, the check is done as part of "reduce slabobj_ext memory with allocation profiling disabled" using slab_obj_ext_has_codetag()
(Harry).
- Move kfence checks earlier in mark_obj_codetag_empty() in Patch 1
(Harry).
- Remove SLAB_MAY_ACCOUNT also for KMALLOC_NO_OBJ_EXT caches and more
precise description in Patch 11 (Hao)
- Add a new cleanup patch 13 that removes also memcg accounting from
kfence allocations, as suggested by Harry.
- Link to v2: https://patch.msgid.link/[email protected]

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/13mm/slab: skip kfence objects in allocation profilingDocumentation/mm/allocation-profiling.rst mm/slub.c+16 -02 replies
2/13mm/slab: remove objs_per_slab()mm/kfence/kfence_test.c mm/slab.h mm/slub.c+2 -91 reply
3/13mm: move struct slabobj_ext to mm/slab.hinclude/linux/memcontrol.h mm/slab.h+13 -130 replies
4/13mm/slab: make slab_obj_ext() determine object indexmm/memcontrol.c mm/slab.h mm/slub.c+21 -320 replies
5/13mm/slab: abstract slabobj_ext.objcg accessmm/kfence/core.c mm/memcontrol.c mm/slab.h +1 more+24 -91 reply
6/13mm/slab: abstract slabobj_ext.ref accessmm/slab.h mm/slub.c+37 -155 replies
7/13mm/slab: replace slab.stride with obj_exts_in_objectmm/slab.h mm/slub.c+41 -452 replies
8/13mm/slab: change struct slabobj_ext to a unionmm/slab.h mm/slub.c+50 -184 replies
9/13mm/slab: introduce slab_obj_ext_has_codetag()mm/slab.h mm/slub.c+46 -02 replies
10/13mm/slab: reduce slabobj_ext memory with allocation profiling disabledmm/slab.h mm/slub.c+4 -13 replies
11/13mm/slab: add cache_ and slab_needs_objcg() helpersinclude/linux/slab.h mm/kfence/core.c mm/slab.h +2 more+61 -81 reply
12/13mm/slab: stop allocating objcg pointers when unnecessarymm/memcontrol.c mm/slab.h mm/slub.c+20 -32 replies
13/13mm/slab, kfence, memcg: completely remove obj_ext for kfence objectsmm/kfence/core.c mm/kfence/kfence.h mm/memcontrol.c +2 more+28 -391 reply

Reviews trailers per patch

All 13 patches have a Reviewed-by or Acked-by.

PatchSuren BaghdasaryanHarry Yoo (Oracle)Hao Li
1/13R-b
2/13R-b R-b R-b R-b
3/13R-b R-b R-b
4/13R-b R-b R-b
5/13R-b R-b R-b
6/13R-b R-b R-b R-b R-b R-b
7/13R-b R-b R-b
8/13R-b R-b R-b
9/13R-b R-b R-b
10/13R-b R-b R-b R-b
11/13R-b R-b
12/13R-b R-b
13/13R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v3 thread

Cover letter [PATCH v3 00/13] mm/slab, alloc_tag: reduce obj_ext memory waste

The recent fixes for objext array handling inspired me to look into this finally. It's been bothering me that the memory usage of struct slabobj_ext depend only on config options and not whether the fields are actually used. So with both CONFIG_MEMCG=y and CONFIG_MEM_ALLOC_PROFILING=y there is always objcg field and codetag_ref field. And thus:

1) Having memory allocation profiling config-enabled but not boot-enabled means wasted memory on unused codetag_refs. This makes it less suitable for a general distro config and the page allocator side doesn't suffer from this, only slab and percpu.

2) Complementary, with memory allocation profiling enabled, there are caches/slabs that don't need the objcg field, so memory is wasted on those.

This series should solve the point 1) fully for slab, pcpuobj_ext handling can be perhaps improved similarly, haven't looked into that.

For 2) it avoids allocating objcg fields for KMALLOC_NORMAL and KMALLOC_NO_OBJ_EXT caches where we know they are not necessary because kmalloc() with __GFP_ACCOUNT will pick a KMALLOC_CGROUP type (except with SLUB_TINY).

The named kmem_caches are tricky. They can be created with SLAB_ACCOUNT and then we know objcg fields are always needed. But also they can be created without SLAB_ACCOUNT and then some allocations have __GFP_ACCOUNT and some not and we don't know that in advance.

This series introduces a SLAB_MAY_ACCOUNT flag that's currently internal only and is applied to all caches (unless kmem accounting is disabled) except KMALLOC_NORMAL (unless that aliases KMALLOC_RECLAIM) and KMALLOC_NO_OBJ_EXT.

As a followup we can make SLAB_MAY_ACCOUNT explicit and add it to to caches where we know __GFP_ACCOUNT is used. Then we could only honour __GFP_ACCOUNT for those, while warning for an unexpected usage elsewhere.

Based on slab/for-next-fixes

Git branch: https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/objext_split

Moderately tested, nothing seems to crash.

To check for regressions, I forward-ported a microbenchmark hacked into slub_kunit that was used to evaluate sheaves. You can find it here along with the series:

https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=objext_split-v3-bench

Tried 3 scenarios, MEMCG and KFENCE were always enabled:
- CONFIG_MEM_ALLOC_PROFILING=n
- CONFIG_MEM_ALLOC_PROFILING=y but _ENABLED_BY_DEFAULT=n
- same but booted with sysctl.vm.mem_profiling=1

The results are quite noisy, but no regression was apparent, except perhaps few percents for the last case. I don't expect it will be visible in any real workloads.

---
Changes in v3:
- Review tags from Harry, Hao, Suren - thanks!
- Patch "skip handle_failed_objexts_alloc() with profiling disabled"
removed, the check is done as part of "reduce slabobj_ext memory with allocation profiling disabled" using slab_obj_ext_has_codetag()
(Harry).
- Move kfence checks earlier in mark_obj_codetag_empty() in Patch 1
(Harry).
- Remove SLAB_MAY_ACCOUNT also for KMALLOC_NO_OBJ_EXT caches and more
precise description in Patch 11 (Hao)
- Add a new cleanup patch 13 that removes also memcg accounting from
kfence allocations, as suggested by Harry.
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- Apply Suren's R-b:, thanks!
- Expanded explanation about no longer accounting KFENCE objects (Suren)
- Also skip them in in mark_obj_codetag_empty() (sashiko)
- Update slab_obj_ext() comments (sashiko)
- Separate slab_obj_ext_objcg() and slab_obj_ext_set_objcg() (Suren)
- Add SLAB_MAY_ACCOUNT internal flag instead of relying on is_kmalloc_normal(); also handle mem_cgroup_kmem_disabled()
- Also fix the SLUB_TINY kmalloc aliasing handling, per Harry.
- Link to v1: https://patch.msgid.link/[email protected]

---
Vlastimil Babka (SUSE) (13):
mm/slab: skip kfence objects in allocation profiling mm/slab: remove objs_per_slab()
mm: move struct slabobj_ext to mm/slab.h
mm/slab: make slab_obj_ext() determine object index mm/slab: abstract slabobj_ext.objcg access
mm/slab: abstract slabobj_ext.ref access
mm/slab: replace slab.stride with obj_exts_in_object mm/slab: change struct slabobj_ext to a union
mm/slab: introduce slab_obj_ext_has_codetag()
mm/slab: reduce slabobj_ext memory with allocation profiling disabled mm/slab: add cache_ and slab_needs_objcg() helpers mm/slab: stop allocating objcg pointers when unnecessary mm/slab, kfence, memcg: completely remove obj_ext for kfence objects

 Documentation/mm/allocation-profiling.rst |   7 ++
 include/linux/memcontrol.h                |  13 --
 include/linux/slab.h                      |   3 +
 mm/kfence/core.c                          |  12 --
 mm/kfence/kfence.h                        |   3 -
 mm/kfence/kfence_test.c                   |   2 +-
 mm/memcontrol.c                           |  40 ++++---
 mm/slab.h                                 | 189 ++++++++++++++++++++++++------
 mm/slab_common.c                          |  26 +++-
 mm/slub.c                                 | 184 ++++++++++++++++++-----------
 10 files changed, 325 insertions(+), 154 deletions(-)

---
base-commit: 5ff172f6c94d282d83cb88bdfec5f647ad9c6105
change-id: 20260714-b4-objext_split-da82426257d5