tools/cgroup/memcg_slabinfo.py is broken
Subsystems (2)
Dear all,
commit 07f910f9b729 ("mm: Remove slab from struct page") broke tools/cgroup/memcg_slabinfo.py script.
It was used as replacement of /psys/fs/cgroup/memory/memory.kmem.slabinfo and was used to monitor slab objects related to specified cgroup
# ./memcg_slabinfo.py /sys/fs/cgroup/memory/lxc.payload.c9s/
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
Traceback (most recent call last):
File "/usr/bin/drgn", line 33, in <module>
sys.exit(load_entry_point('drgn==0.0.16', 'console_scripts', 'drgn')())
File "/usr/lib64/python3.10/site-packages/drgn/internal/cli.py", line 133, in main
runpy.run_path(args.script[0], init_globals=init_globals, run_name="__main__")
File "/usr/lib64/python3.10/runpy.py", line 269, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib64/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib64/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "./memcg_slabinfo.py", line 226, in <module>
main()
File "./memcg_slabinfo.py", line 199, in main
cache = page.slab_cache
AttributeError: 'struct page' has no member 'slab_cache'Any ideas how to repair it ?
Thank you,
Vasily Averin
Quoted 26 lines · Vasily Averin
Dear all,
commit
07f910f9b729("mm: Remove slab from struct page") broketools/cgroup/memcg_slabinfo.pyscript.
It was used as replacement of /psys/fs/cgroup/memory/memory.kmem.slabinfo and was used to monitor slab objects related to specified cgroup# ./memcg_slabinfo.py /sys/fs/cgroup/memory/lxc.payload.c9s/ # name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail> Traceback (most recent call last): File "/usr/bin/drgn", line 33, in <module> sys.exit(load_entry_point('drgn==0.0.16', 'console_scripts', 'drgn')()) File "/usr/lib64/python3.10/site-packages/drgn/internal/cli.py", line 133, in main runpy.run_path(args.script[0], init_globals=init_globals, run_name="__main__") File "/usr/lib64/python3.10/runpy.py", line 269, in run_path return _run_module_code(code, init_globals, run_name, File "/usr/lib64/python3.10/runpy.py", line 96, in _run_module_code _run_code(code, mod_globals, init_globals, File "/usr/lib64/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "./memcg_slabinfo.py", line 226, in <module> main() File "./memcg_slabinfo.py", line 199, in main cache = page.slab_cache AttributeError: 'struct page' has no member 'slab_cache'
Quoted 4 lines
Any ideas how to repair it ?
Thank you,
Vasily Averin
Thanks for the report, Vasily!
I'll take a look, we need to update the script according to the latest slab/page/folio changes.
Quoted 29 lines · Roman Gushchin
Quoted 28 lines · Vasily Averin
Dear all,
commit
07f910f9b729("mm: Remove slab from struct page") broketools/cgroup/memcg_slabinfo.pyscript.
It was used as replacement of /psys/fs/cgroup/memory/memory.kmem.slabinfo and was used to monitor slab objects related to specified cgroup# ./memcg_slabinfo.py /sys/fs/cgroup/memory/lxc.payload.c9s/ # name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail> Traceback (most recent call last): File "/usr/bin/drgn", line 33, in <module> sys.exit(load_entry_point('drgn==0.0.16', 'console_scripts', 'drgn')()) File "/usr/lib64/python3.10/site-packages/drgn/internal/cli.py", line 133, in main runpy.run_path(args.script[0], init_globals=init_globals, run_name="__main__") File "/usr/lib64/python3.10/runpy.py", line 269, in run_path return _run_module_code(code, init_globals, run_name, File "/usr/lib64/python3.10/runpy.py", line 96, in _run_module_code _run_code(code, mod_globals, init_globals, File "/usr/lib64/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "./memcg_slabinfo.py", line 226, in <module> main() File "./memcg_slabinfo.py", line 199, in main cache = page.slab_cache AttributeError: 'struct page' has no member 'slab_cache'Any ideas how to repair it ?
Quoted 7 lines
Quoted 2 lines
Thank you,
Vasily AverinThanks for the report, Vasily!
I'll take a look, we need to update the script according to the latest slab/page/folio changes.
I've workarounded it temporally by the attached patch,.
tools/cgroup/memcg_slabinfo.py +5 -3
@@ -11,7 +11,7 @@ from drgn.helpers.linux import list_for_each_entry, list_empty from drgn.helpers.linux import for_each_page from drgn.helpers.linux.cpumask import for_each_online_cpu from drgn.helpers.linux.percpu import per_cpu_ptr -from drgn import container_of, FaultError, Object +from drgn import container_of, FaultError, Object, cast DESC = """
@@ -77,7 +77,8 @@ OO_MASK = ((1 << OO_SHIFT) - 1) def count_free(page): - return page.objects - page.inuse + slab = cast("struct slab *", page) + return slab.objects - slab.inuse def slub_get_slabinfo(s, cfg):
@@ -196,7 +197,8 @@ OO_MASK = ((1 << OO_SHIFT) - 1) objcg_vec_raw = page.memcg_data.value_() if objcg_vec_raw == 0: continue - cache = page.slab_cache + slab = cast("struct slab *", page) + cache = slab.slab_cache if not cache: continue addr = cache.value_()
Quoted 4 lines · Vasily Averin
One or more of the originally attached files triggered the rule module.access.rule.exestrip_notify
The following attachments were deleted from the original message.
diff-memcg-slabinfo.py
Sorry, I haven't received it.
Anyway, can you, please, test the following patch?
Thanks!
--
From b649752a0b6de605ce4ba4be73ddf99ea7dd1d5c Mon Sep 17 00:00:00 2001
From: Roman Gushchin <[email protected]>
Date: Wed, 16 Feb 2022 12:43:30 -0800
Subject: [PATCH] tools/cgroup/slabinfo: update to work with struct slab
After the introduction of the dedicated struct slab to describe slab pages by commit d122019bf061 ("mm: Split slab into its own type") and the following removal of the corresponding struct page's fields by commit 07f910f9b729 ("mm: Remove slab from struct page") the memcg_slabinfo tool broke. An attempt to run it produces a trace like this:
Traceback (most recent call last):
File "/usr/bin/drgn", line 33, in <module>
sys.exit(load_entry_point('drgn==0.0.16', 'console_scripts', 'drgn')())
File "/usr/lib64/python3.9/site-packages/drgn/internal/cli.py", line 133, in main
runpy.run_path(args.script[0], init_globals=init_globals, run_name="__main__")
File "/usr/lib64/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "memcg_slabinfo.py", line 226, in <module>
main() File "memcg_slabinfo.py", line 199, in main
cache = page.slab_cache
AttributeError: 'struct page' has no member 'slab_cache'
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
cred_jar 776 776 192 21 1 : tunables 0 0 0 : slabdata 547 547 0 kmalloc-cg-32 6 6 32 128 1 : tunables 0 0 0 : slabdata 9 9 0 files_cache 3 3 832 39 8 : tunables 0 0 0 : slabdata 8 8 0 kmalloc-cg-512 1 1 512 32 4 : tunables 0 0 0 : slabdata 10 10 0 task_struct 10 10 6720 4 8 : tunables 0 0 0 : slabdata 63 63 0 mm_struct 3 3 1664 19 8 : tunables 0 0 0 : slabdata 9 9 0 kmalloc-cg-16 1 1 16 256 1 : tunables 0 0 0 : slabdata 8 8 0 pde_opener 1 1 40 102 1 : tunables 0 0 0 : slabdata 8 8 0 anon_vma_chain 375 375 64 64 1 : tunables 0 0 0 : slabdata 81 81 0 radix_tree_node 3 3 584 28 4 : tunables 0 0 0 : slabdata 419 419 0 dentry 98 98 312 26 2 : tunables 0 0 0 : slabdata 1420 1420 0 btrfs_inode 3 3 2368 13 8 : tunables 0 0 0 : slabdata 730 730 0 signal_cache 3 3 1600 20 8 : tunables 0 0 0 : slabdata 17 17 0 sighand_cache 3 3 2240 14 8 : tunables 0 0 0 : slabdata 20 20 0 filp 90 90 512 32 4 : tunables 0 0 0 : slabdata 95 95 0 anon_vma 214 214 200 20 1 : tunables 0 0 0 : slabdata 162 162 0 kmalloc-cg-1k 1 1 1024 32 8 : tunables 0 0 0 : slabdata 22 22 0 pid 10 10 256 32 2 : tunables 0 0 0 : slabdata 14 14 0 kmalloc-cg-64 2 2 64 64 1 : tunables 0 0 0 : slabdata 8 8 0 kmalloc-cg-96 3 3 96 42 1 : tunables 0 0 0 : slabdata 8 8 0 sock_inode_cache 5 5 1408 23 8 : tunables 0 0 0 : slabdata 29 29 0 UNIX 7 7 1920 17 8 : tunables 0 0 0 : slabdata 21 21 0 inode_cache 36 36 1152 28 8 : tunables 0 0 0 : slabdata 680 680 0 proc_inode_cache 26 26 1224 26 8 : tunables 0 0 0 : slabdata 64 64 0 kmalloc-cg-2k 2 2 2048 16 8 : tunables 0 0 0 : slabdata 9 9 0
--- tools/cgroup/memcg_slabinfo.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
tools/cgroup/memcg_slabinfo.py +6 -4
@@ -11,7 +11,7 @@ from drgn.helpers.linux import list_for_each_entry, list_empty from drgn.helpers.linux import for_each_page from drgn.helpers.linux.cpumask import for_each_online_cpu from drgn.helpers.linux.percpu import per_cpu_ptr -from drgn import container_of, FaultError, Object +from drgn import container_of, FaultError, Object, cast DESC = """
@@ -77,7 +77,8 @@ def count_partial(n, fn): def count_free(page): - return page.objects - page.inuse + slab = cast('struct slab *', page) + return slab.objects - slab.inuse def slub_get_slabinfo(s, cfg):
@@ -193,10 +194,11 @@ def main(): # look over all slab pages, belonging to non-root memcgs # and look for objects belonging to the given memory cgroup for page in for_each_slab_page(prog): - objcg_vec_raw = page.memcg_data.value_() + slab = cast('struct slab *', page) + objcg_vec_raw = slab.memcg_data.value_() if objcg_vec_raw == 0: continue - cache = page.slab_cache + cache = slab.slab_cache if not cache: continue addr = cache.value_()
Signature · 1 line
2.34.1
Quoted 2 lines · Roman Gushchin
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
This feels like a quick fix, and not really correct.
tools/cgroup/memcg_slabinfo.py def count_partial(n, fn): · line 81
Quoted 7 lines
tools/cgroup/memcg_slabinfo.py +2
-1@@ -77,7 +77,8 @@ def count_partial(n, fn): def count_free(page): - return page.objects - page.inuse + slab = cast('struct slab *', page) + return slab.objects - slab.inuse
count_free() should take a slab, not a page.
tools/cgroup/memcg_slabinfo.py def main(): · line 196
Quoted 5 lines
tools/cgroup/memcg_slabinfo.py +0
-0def slub_get_slabinfo(s, cfg):@@ -193,10 +194,11 @@ def main(): # look over all slab pages, belonging to non-root memcgs # and look for objects belonging to the given memory cgroup for page in for_each_slab_page(prog):
for_each_slab_page() should be renamed for_each_slab(). It should return a slab, not a page. And it should definitely skip over tail pages (it works today by coincidence because tail pages do not have PG_slab set).
count_partial() should use struct slab, and slab_list, not lru.
... I think that's it. But I'm no pythonist, much less dragoneer.
Quoted 5 lines · Matthew Wilcox
Quoted 2 lines · Roman Gushchin
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
This feels like a quick fix, and not really correct.
Do you mind to provide a correct version?
Thanks!
Quoted 8 lines · Roman Gushchin
Quoted 5 lines · Matthew Wilcox
Quoted 2 lines · Roman Gushchin
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
This feels like a quick fix, and not really correct.
Do you mind to provide a correct version?
I know nothing about python, nor how to even run this script. But this is the kind of thing I was thinking about.
I didn't do the 'skip over tail pages' because I have no idea how you'd tell for_each_page() to do that. I don't know where to get drgn.helpers.linux from, so I can't look at the implementation. Maybe it already does that.
tools/cgroup/memcg_slabinfo.py +14 -14
@@ -11,7 +11,7 @@ from drgn.helpers.linux import list_for_each_entry, list_empty from drgn.helpers.linux import for_each_page from drgn.helpers.linux.cpumask import for_each_online_cpu from drgn.helpers.linux.percpu import per_cpu_ptr -from drgn import container_of, FaultError, Object +from drgn import container_of, FaultError, Object, cast DESC = """
@@ -69,15 +69,15 @@ def oo_objects(s): def count_partial(n, fn): - nr_pages = 0 - for page in list_for_each_entry('struct page', n.partial.address_of_(), - 'lru'): - nr_pages += fn(page) - return nr_pages + nr_objs = 0 + for slab in list_for_each_entry('struct slab', n.partial.address_of_(), + 'slab_list'): + nr_objs += fn(slab) + return nr_objs -def count_free(page): - return page.objects - page.inuse +def count_free(slab): + return slab.objects - slab.inuse def slub_get_slabinfo(s, cfg):
@@ -145,14 +145,14 @@ def detect_kernel_config(): return cfg -def for_each_slab_page(prog): +def for_each_slab(prog): PGSlab = 1 << prog.constant('PG_slab') PGHead = 1 << prog.constant('PG_head') for page in for_each_page(prog): try: if page.flags.value_() & PGSlab: - yield page + yield cast('struct slab *', page) except FaultError: pass
@@ -190,13 +190,13 @@ def main(): 'list'): obj_cgroups.add(ptr.value_()) - # look over all slab pages, belonging to non-root memcgs + # look over all slabs, belonging to non-root memcgs # and look for objects belonging to the given memory cgroup - for page in for_each_slab_page(prog): - objcg_vec_raw = page.memcg_data.value_() + for slab in for_each_slab(prog): + objcg_vec_raw = slab.memcg_data.value_() if objcg_vec_raw == 0: continue - cache = page.slab_cache + cache = slab.slab_cache if not cache: continue addr = cache.value_()
Quoted 10 lines · Matthew Wilcox
Quoted 8 lines · Roman Gushchin
Quoted 5 lines · Matthew Wilcox
Quoted 2 lines · Roman Gushchin
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
This feels like a quick fix, and not really correct.
Do you mind to provide a correct version?
Quoted 7 lines
I know nothing about python, nor how to even run this script. But this is the kind of thing I was thinking about.
I didn't do the 'skip over tail pages' because I have no idea how you'd tell
for_each_page()to do that. I don't know where to get drgn.helpers.linux from, so I can't look at the implementation. Maybe it already does that.
Is there an actual plan to set the slab flag for tail pages? If not, why to check both? It's fairly expensive already, so I'd not add any additional checks if there is no strict necessity.
Quoted 26 lines
Signed-off-by Matthew Wilcox (Oracle) <[email protected]>tools/cgroup/memcg_slabinfo.py +2
-6@@ -11,7 +11,7 @@ from drgn.helpers.linux import list_for_each_entry, list_empty from drgn.helpers.linux import for_each_page from drgn.helpers.linux.cpumask import for_each_online_cpu from drgn.helpers.linux.percpu import per_cpu_ptr -from drgn import container_of, FaultError, Object +from drgn import container_of, FaultError, Object, cast DESC = """@@ -69,15 +69,15 @@ def oo_objects(s): def count_partial(n, fn): - nr_pages = 0 - for page in list_for_each_entry('struct page', n.partial.address_of_(), - 'lru'): - nr_pages += fn(page) - return nr_pages + nr_objs = 0
tools/cgroup/memcg_slabinfo.py def oo_objects(s): · line 80
Quoted 10 lines
tools/cgroup/memcg_slabinfo.py +6
-2+ for slab in list_for_each_entry('struct slab', n.partial.address_of_(), + 'slab_list'): + nr_objs += fn(slab) + return nr_objs -def count_free(page): - return page.objects - page.inuse +def count_free(slab): + return slab.objects - slab.inuse
This part looks good to me.
tools/cgroup/memcg_slabinfo.py def detect_kernel_config(): · line 148
Quoted 9 lines
tools/cgroup/memcg_slabinfo.py +1
-1def slub_get_slabinfo(s, cfg):@@ -145,14 +145,14 @@ def detect_kernel_config(): return cfg -def for_each_slab_page(prog): +def for_each_slab(prog):
Here I'd keep _page (or _folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).
Quoted 12 lines
tools/cgroup/memcg_slabinfo.py +1
-1PGSlab = 1 << prog.constant('PG_slab') PGHead = 1 << prog.constant('PG_head') for page in for_each_page(prog): try: if page.flags.value_() & PGSlab: - yield page + yield cast('struct slab *', page) except FaultError: pass
tools/cgroup/memcg_slabinfo.py def main(): · line 196
Quoted 10 lines
tools/cgroup/memcg_slabinfo.py +3
-3@@ -190,13 +190,13 @@ def main(): 'list'): obj_cgroups.add(ptr.value_()) - # look over all slab pages, belonging to non-root memcgs + # look over all slabs, belonging to non-root memcgs # and look for objects belonging to the given memory cgroup - for page in for_each_slab_page(prog): - objcg_vec_raw = page.memcg_data.value_() + for slab in for_each_slab(prog): + objcg_vec_raw = slab.memcg_data.value_()
So I'd go with something like this:
for folio in for_each_slab_folio(prog):
slab = cast('struct slab *', folio)
...
tools/cgroup/memcg_slabinfo.py def main(): · line 202
Quoted 7 lines
tools/cgroup/memcg_slabinfo.py +1
-1if objcg_vec_raw == 0: continue - cache = page.slab_cache + cache = slab.slab_cache if not cache: continue addr = cache.value_()
But I'm fine with either version.
Thanks!
Quoted 14 lines · Roman Gushchin
Quoted 13 lines · Matthew Wilcox
Quoted 8 lines · Roman Gushchin
Quoted 5 lines · Matthew Wilcox
Quoted 2 lines · Roman Gushchin
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
This feels like a quick fix, and not really correct.
Do you mind to provide a correct version?
I know nothing about python, nor how to even run this script. But this is the kind of thing I was thinking about.
Quoted 8 lines
Quoted 4 lines
I didn't do the 'skip over tail pages' because I have no idea how you'd tell
for_each_page()to do that. I don't know where to get drgn.helpers.linux from, so I can't look at the implementation. Maybe it already does that.Is there an actual plan to set the slab flag for tail pages? If not, why to check both? It's fairly expensive already, so I'd not add any additional checks if there is no strict necessity.
The plan to set the slab flag on tail pages is probably about five years away. There are a lot of moving parts before we get there. I think you could reduce the cost if we had a for_each_folio() loop. Most code doesn't really want to look at every page, so having a shorter list (and iterating over each page within a folio for the few places that really do want to see every page) would be a good tradeoff.
Quoted 10 lines
tools/cgroup/memcg_slabinfo.py def detect_kernel_config(): · line 148
Quoted 6 lines
tools/cgroup/memcg_slabinfo.py +1
-1@@ -145,14 +145,14 @@ def detect_kernel_config(): return cfg -def for_each_slab_page(prog): +def for_each_slab(prog):Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).
But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Quoted 1 line · Matthew Wilcox
[...]
Quoted 8 lines
Quoted 3 lines
Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Is there enough space in the struct slab for that list or do you mean something different?
Quoted 7 lines · Shakeel Butt
Quoted 1 line · Matthew Wilcox
[...]
Quoted 4 lines
Quoted 3 lines
Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).
Quoted 7 lines
Quoted 4 lines
But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Is there enough space in the struct slab for that list or do you mean something different?
Well, I don't know what data structure your for_each_page() uses, so I don't know what to suggest exactly.
If it's an iteration over the entirety of memmap, then maybe you'd rather iterate over each kmem_cache like /proc/slabinfo does?
Quoted 13 lines · Matthew Wilcox
Quoted 12 lines · Shakeel Butt
Quoted 1 line · Matthew Wilcox
[...]
Quoted 8 lines
Quoted 3 lines
Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Quoted 8 lines
Quoted 2 lines
Is there enough space in the struct slab for that list or do you mean something different?
Well, I don't know what data structure your
for_each_page()uses, so I don't know what to suggest exactly.If it's an iteration over the entirety of memmap, then maybe you'd rather iterate over each
kmem_cachelike/proc/slabinfodoes?
For SLUB /proc/slabinfo ignores pages on percpu partial lists, which can significantly alter results.
For slabinfo.py the accuracy is more important than performance, at least in my understanding. It's not a tool which a generic linux user would run often.
Quoted 13 lines · Matthew Wilcox
Quoted 12 lines · Shakeel Butt
Quoted 1 line · Matthew Wilcox
[...]
Quoted 8 lines
Quoted 3 lines
Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Quoted 8 lines
Quoted 2 lines
Is there enough space in the struct slab for that list or do you mean something different?
Well, I don't know what data structure your
for_each_page()uses, so I don't know what to suggest exactly.If it's an iteration over the entirety of memmap, then maybe you'd rather iterate over each
kmem_cachelike/proc/slabinfodoes?
Anyway, here is v2. I hope you're fine with it.
--
From 04efbbdcfde4535c4623f2fd6910a505a9c638bb Mon Sep 17 00:00:00 2001
From: Roman Gushchin <[email protected]>
Date: Wed, 16 Feb 2022 12:43:30 -0800
Subject: [PATCH v2] tools/cgroup/slabinfo: update to work with struct slab
After the introduction of the dedicated struct slab to describe slab pages by commit d122019bf061 ("mm: Split slab into its own type") and the following removal of the corresponding struct page's fields by commit 07f910f9b729 ("mm: Remove slab from struct page") the memcg_slabinfo tool broke. An attempt to run it produces a trace like this:
Traceback (most recent call last):
File "/usr/bin/drgn", line 33, in <module>
sys.exit(load_entry_point('drgn==0.0.16', 'console_scripts', 'drgn')())
File "/usr/lib64/python3.9/site-packages/drgn/internal/cli.py", line 133, in main
runpy.run_path(args.script[0], init_globals=init_globals, run_name="__main__")
File "/usr/lib64/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "memcg_slabinfo.py", line 226, in <module>
main() File "memcg_slabinfo.py", line 199, in main
cache = page.slab_cache
AttributeError: 'struct page' has no member 'slab_cache'
The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
cred_jar 776 776 192 21 1 : tunables 0 0 0 : slabdata 547 547 0 kmalloc-cg-32 6 6 32 128 1 : tunables 0 0 0 : slabdata 9 9 0 files_cache 3 3 832 39 8 : tunables 0 0 0 : slabdata 8 8 0 kmalloc-cg-512 1 1 512 32 4 : tunables 0 0 0 : slabdata 10 10 0 task_struct 10 10 6720 4 8 : tunables 0 0 0 : slabdata 63 63 0 mm_struct 3 3 1664 19 8 : tunables 0 0 0 : slabdata 9 9 0 kmalloc-cg-16 1 1 16 256 1 : tunables 0 0 0 : slabdata 8 8 0 pde_opener 1 1 40 102 1 : tunables 0 0 0 : slabdata 8 8 0 anon_vma_chain 375 375 64 64 1 : tunables 0 0 0 : slabdata 81 81 0 radix_tree_node 3 3 584 28 4 : tunables 0 0 0 : slabdata 419 419 0 dentry 98 98 312 26 2 : tunables 0 0 0 : slabdata 1420 1420 0 btrfs_inode 3 3 2368 13 8 : tunables 0 0 0 : slabdata 730 730 0 signal_cache 3 3 1600 20 8 : tunables 0 0 0 : slabdata 17 17 0 sighand_cache 3 3 2240 14 8 : tunables 0 0 0 : slabdata 20 20 0 filp 90 90 512 32 4 : tunables 0 0 0 : slabdata 95 95 0 anon_vma 214 214 200 20 1 : tunables 0 0 0 : slabdata 162 162 0 kmalloc-cg-1k 1 1 1024 32 8 : tunables 0 0 0 : slabdata 22 22 0 pid 10 10 256 32 2 : tunables 0 0 0 : slabdata 14 14 0 kmalloc-cg-64 2 2 64 64 1 : tunables 0 0 0 : slabdata 8 8 0 kmalloc-cg-96 3 3 96 42 1 : tunables 0 0 0 : slabdata 8 8 0 sock_inode_cache 5 5 1408 23 8 : tunables 0 0 0 : slabdata 29 29 0 UNIX 7 7 1920 17 8 : tunables 0 0 0 : slabdata 21 21 0 inode_cache 36 36 1152 28 8 : tunables 0 0 0 : slabdata 680 680 0 proc_inode_cache 26 26 1224 26 8 : tunables 0 0 0 : slabdata 64 64 0 kmalloc-cg-2k 2 2 2048 16 8 : tunables 0 0 0 : slabdata 9 9 0
v2: change naming and count_partial()/count_free()/for_each_slab()
signatures to work with slabs, suggested by Matthew Wilcox
--- tools/cgroup/memcg_slabinfo.py | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-)
tools/cgroup/memcg_slabinfo.py +15 -15
@@ -11,7 +11,7 @@ from drgn.helpers.linux import list_for_each_entry, list_empty from drgn.helpers.linux import for_each_page from drgn.helpers.linux.cpumask import for_each_online_cpu from drgn.helpers.linux.percpu import per_cpu_ptr -from drgn import container_of, FaultError, Object +from drgn import container_of, FaultError, Object, cast DESC = """
@@ -69,15 +69,15 @@ def oo_objects(s): def count_partial(n, fn): - nr_pages = 0 - for page in list_for_each_entry('struct page', n.partial.address_of_(), - 'lru'): - nr_pages += fn(page) - return nr_pages + nr_objs = 0 + for slab in list_for_each_entry('struct slab', n.partial.address_of_(), + 'slab_list'): + nr_objs += fn(slab) + return nr_objs -def count_free(page): - return page.objects - page.inuse +def count_free(slab): + return slab.objects - slab.inuse def slub_get_slabinfo(s, cfg):
@@ -145,14 +145,14 @@ def detect_kernel_config(): return cfg -def for_each_slab_page(prog): +def for_each_slab(prog): PGSlab = 1 << prog.constant('PG_slab') PGHead = 1 << prog.constant('PG_head') for page in for_each_page(prog): try: if page.flags.value_() & PGSlab: - yield page + yield cast('struct slab *', page) except FaultError: pass
@@ -190,13 +190,13 @@ def main(): 'list'): obj_cgroups.add(ptr.value_()) - # look over all slab pages, belonging to non-root memcgs - # and look for objects belonging to the given memory cgroup - for page in for_each_slab_page(prog): - objcg_vec_raw = page.memcg_data.value_() + # look over all slab folios and look for objects belonging + # to the given memory cgroup + for slab in for_each_slab(prog): + objcg_vec_raw = slab.memcg_data.value_() if objcg_vec_raw == 0: continue - cache = page.slab_cache + cache = slab.slab_cache if not cache: continue addr = cache.value_()
Signature · 1 line
2.34.1
Vasily Averin
Tested-byQuoted 17 lines · Roman Gushchin
Quoted 16 lines · Matthew Wilcox
Quoted 14 lines · Shakeel Butt
Quoted 1 line · Matthew Wilcox
[...]
Quoted 8 lines
Quoted 3 lines
Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Is there enough space in the struct slab for that list or do you mean something different?
Quoted 7 lines
Quoted 5 lines
Well, I don't know what data structure your
for_each_page()uses, so I don't know what to suggest exactly.If it's an iteration over the entirety of memmap, then maybe you'd rather iterate over each
kmem_cachelike/proc/slabinfodoes?Anyway, here is v2. I hope you're fine with it.
It works well on my test node,
Andrew,
this script was created in August 2020, broken in few months, repaired in April 2021, then broken again in October 2021 and I hope will be repaired soon.
I think it would be useful to add smoke test with this script into checks of some build robots.
Could you please advise how to do it?
Thank you,
Vasily Averin
Quoted 138 lines
Quoted 1 line
From
04efbbdcfde4535c4623f2fd6910a505a9c638bbMon Sep 17 00:00:00 2001From: Roman Gushchin <[email protected]>
Date: Wed, 16 Feb 2022 12:43:30 -0800
Subject: [PATCH v2]tools/cgroup/slabinfo: update to work with struct slabAfter the introduction of the dedicated struct slab to describe slab pages by commit
d122019bf061("mm: Split slab into its own type") and the following removal of the corresponding struct page's fields by commit07f910f9b729("mm: Remove slab from struct page") thememcg_slabinfotool broke. An attempt to run it produces a trace like this:
Traceback (most recent call last):File "/usr/bin/drgn", line 33, in <module> sys.exit(load_entry_point('drgn==0.0.16', 'console_scripts', 'drgn')()) File "/usr/lib64/python3.9/site-packages/drgn/internal/cli.py", line 133, in main runpy.run_path(args.script[0], init_globals=init_globals, run_name="__main__") File "/usr/lib64/python3.9/runpy.py", line 268, in run_path return _run_module_code(code, init_globals, run_name, File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code _run_code(code, mod_globals, init_globals, File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "memcg_slabinfo.py", line 226, in <module> main() File "memcg_slabinfo.py", line 199, in main cache = page.slab_cacheAttributeError: 'struct page' has no member '
slab_cache'The problem can be fixed by explicitly casting struct page * to struct slab * for slab pages. The tools works as expected with this fix, e.g.:
cred_jar 776 776 192 21 1 : tunables 0 0 0 : slabdata 547 547 0 kmalloc-cg-32 6 6 32 128 1 : tunables 0 0 0 : slabdata 9 9 0 files_cache 3 3 832 39 8 : tunables 0 0 0 : slabdata 8 8 0 kmalloc-cg-512 1 1 512 32 4 : tunables 0 0 0 : slabdata 10 10 0 task_struct 10 10 6720 4 8 : tunables 0 0 0 : slabdata 63 63 0 mm_struct 3 3 1664 19 8 : tunables 0 0 0 : slabdata 9 9 0 kmalloc-cg-16 1 1 16 256 1 : tunables 0 0 0 : slabdata 8 8 0 pde_opener 1 1 40 102 1 : tunables 0 0 0 : slabdata 8 8 0 anon_vma_chain 375 375 64 64 1 : tunables 0 0 0 : slabdata 81 81 0 radix_tree_node 3 3 584 28 4 : tunables 0 0 0 : slabdata 419 419 0 dentry 98 98 312 26 2 : tunables 0 0 0 : slabdata 1420 1420 0 btrfs_inode 3 3 2368 13 8 : tunables 0 0 0 : slabdata 730 730 0 signal_cache 3 3 1600 20 8 : tunables 0 0 0 : slabdata 17 17 0 sighand_cache 3 3 2240 14 8 : tunables 0 0 0 : slabdata 20 20 0 filp 90 90 512 32 4 : tunables 0 0 0 : slabdata 95 95 0 anon_vma 214 214 200 20 1 : tunables 0 0 0 : slabdata 162 162 0 kmalloc-cg-1k 1 1 1024 32 8 : tunables 0 0 0 : slabdata 22 22 0 pid 10 10 256 32 2 : tunables 0 0 0 : slabdata 14 14 0 kmalloc-cg-64 2 2 64 64 1 : tunables 0 0 0 : slabdata 8 8 0 kmalloc-cg-96 3 3 96 42 1 : tunables 0 0 0 : slabdata 8 8 0 sock_inode_cache 5 5 1408 23 8 : tunables 0 0 0 : slabdata 29 29 0 UNIX 7 7 1920 17 8 : tunables 0 0 0 : slabdata 21 21 0 inode_cache 36 36 1152 28 8 : tunables 0 0 0 : slabdata 680 680 0 proc_inode_cache 26 26 1224 26 8 : tunables 0 0 0 : slabdata 64 64 0 kmalloc-cg-2k 2 2 2048 16 8 : tunables 0 0 0 : slabdata 9 9 0v2: change naming and count_partial()/count_free()/for_each_slab()
signatures to work with slabs, suggested by Matthew WilcoxSigned-off-by Roman Gushchin <[email protected]>Reported-by Vasily Averin <[email protected]>--- tools/cgroup/memcg_slabinfo.py | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-)tools/cgroup/memcg_slabinfo.py +15
-15@@ -11,7 +11,7 @@ from drgn.helpers.linux import list_for_each_entry, list_empty from drgn.helpers.linux import for_each_page from drgn.helpers.linux.cpumask import for_each_online_cpu from drgn.helpers.linux.percpu import per_cpu_ptr -from drgn import container_of, FaultError, Object +from drgn import container_of, FaultError, Object, cast DESC = """@@ -69,15 +69,15 @@ def oo_objects(s): def count_partial(n, fn): - nr_pages = 0 - for page in list_for_each_entry('struct page', n.partial.address_of_(), - 'lru'): - nr_pages += fn(page) - return nr_pages + nr_objs = 0 + for slab in list_for_each_entry('struct slab', n.partial.address_of_(), + 'slab_list'): + nr_objs += fn(slab) + return nr_objs -def count_free(page): - return page.objects - page.inuse +def count_free(slab): + return slab.objects - slab.inuse def slub_get_slabinfo(s, cfg):@@ -145,14 +145,14 @@ def detect_kernel_config(): return cfg -def for_each_slab_page(prog): +def for_each_slab(prog): PGSlab = 1 << prog.constant('PG_slab') PGHead = 1 << prog.constant('PG_head') for page in for_each_page(prog): try: if page.flags.value_() & PGSlab: - yield page + yield cast('struct slab *', page) except FaultError: pass@@ -190,13 +190,13 @@ def main(): 'list'): obj_cgroups.add(ptr.value_()) - # look over all slab pages, belonging to non-root memcgs - # and look for objects belonging to the given memory cgroup - for page in for_each_slab_page(prog): - objcg_vec_raw = page.memcg_data.value_() + # look over all slab folios and look for objects belonging + # to the given memory cgroup + for slab in for_each_slab(prog): + objcg_vec_raw = slab.memcg_data.value_() if objcg_vec_raw == 0: continue - cache = page.slab_cache + cache = slab.slab_cache if not cache: continue addr = cache.value_()
Quoted 21 lines · Vasily Averin
Quoted 20 lines · Roman Gushchin
Quoted 19 lines · Matthew Wilcox
Quoted 7 lines · Shakeel Butt
Quoted 1 line · Matthew Wilcox
[...]
Quoted 4 lines
Quoted 3 lines
Here I'd keep
_page(or_folio, if you want), because it makes it clear that the function goes over all pages (and is expensive therefore).Quoted 7 lines
Quoted 4 lines
But there's no intrinsic reason that it should. If that's a performance problem, you could keep slabs on a separate list from, eg, file or anon memory.
Is there enough space in the struct slab for that list or do you mean something different?
Well, I don't know what data structure your
for_each_page()uses, so I don't know what to suggest exactly.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
If it's an iteration over the entirety of memmap, then maybe you'd rather iterate over each
kmem_cachelike/proc/slabinfodoes?Anyway, here is v2. I hope you're fine with it.
It works well on my test node,
Tested-by Vasily Averin <[email protected]>
Thanks all for fixing and testing!
Quoted 4 lines
Andrew,
this script was created in August 2020, broken in few months, repaired in April 2021,
then broken again in October 2021 and I hope will be repaired soon.
Since it was me who broke it now in 5.17-rc1 and it's slab related, I'd take the fix to slab tree and send to Linus so it's in 5.17 final.
Quoted 4 lines
I think it would be useful to add smoke test with this script into checks of some build robots.
Could you please advise how to do it?
I guess ask the lkp maintainers?
BTW I'm a bit skeptical in general about maintaining drgn (or gdb) based python scriplets within the kernel tree, that are supposed to work only with that specific version. The model of the 'crash' tool which supports a range of kernels seems more practical and sustainable to me.
Matthew Wilcox
Reviewed-byQuoted 9 lines · Roman Gushchin
Quoted 1 line
From
04efbbdcfde4535c4623f2fd6910a505a9c638bbMon Sep 17 00:00:00 2001From: Roman Gushchin <[email protected]>
Date: Wed, 16 Feb 2022 12:43:30 -0800
Subject: [PATCH v2]tools/cgroup/slabinfo: update to work with struct slabAfter the introduction of the dedicated struct slab to describe slab pages by commit
d122019bf061("mm: Split slab into its own type") and the following removal of the corresponding struct page's fields by commit07f910f9b729("mm: Remove slab from struct page") the
[...]
Quoted 6 lines
v2: change naming and count_partial()/count_free()/for_each_slab()
signatures to work with slabs, suggested by Matthew WilcoxSigned-off-by Roman Gushchin <[email protected]>Reported-by Vasily Averin <[email protected]>