alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled
Subsystems (4)
Suren Baghdasaryan
maintainerPATCH 1/2 alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled
When memory allocation profiling is disabled there is no need to update current->alloc_tag and these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.
--- include/linux/alloc_tag.h | 11 ++++++++--- lib/alloc_tag.c | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-)
include/linux/alloc_tag.h +8 -3
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}#define alloc_hooks_tag (_tag, _do_alloc) \ ({ \ - struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag); \ - typeof(_do_alloc) _res = _do_alloc; \ - alloc_tag_restore(_tag, _old); \ + typeof(_do_alloc) _res; \ + if (mem_alloc_profiling_enabled()) { \ + struct alloc_tag * __maybe_unused _old; \ + _old = alloc_tag_save(_tag); \ + _res = _do_alloc; \ + alloc_tag_restore(_tag, _old); \ + } else \ + _res = _do_alloc; \ _res; \ })
lib/alloc_tag.c +2 -0
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);DEFINE_STATIC_KEY_MAYBE (CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT , mem_alloc_profiling_key); +EXPORT_SYMBOL (mem_alloc_profiling_key); +DEFINE_STATIC_KEY_FALSE (mem_profiling_compressed);struct alloc_tag_kernel_section kernel_tags = {NULL ,0 };
base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Suren Baghdasaryan
maintainerPATCH 2/2 alloc_tag: skip pgalloc_tag_swap if profiling is disabled
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
--- lib/alloc_tag.c | 3 +++ 1 file changed, 3 insertions(+)
lib/alloc_tag.c +3 -0
@@ -197,6 +197,9 @@ void pgalloc_tag_swap(struct folio *new, struct folio *old)union codetag_ref ref_old, ref_new;struct alloc_tag *tag_old, *tag_new; +if (!mem_alloc_profiling_enabled ()) +return ; + tag_old =pgalloc_tag_get (&old->page);if (!tag_old)return ;
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Andrew Morton
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>
Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
Suren Baghdasaryan
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 6 lines · Andrew Morton
Quoted 3 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Quoted 7 lines
Quoted 3 lines
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled. The second one I couldn't really measure, so I think it's not as important. Thanks!
Quoted 1 line
Andrew Morton
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 11 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>
Quoted 8 lines
Quoted 3 lines
Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
The first patch ("
alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.
um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.
Suren Baghdasaryan
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
Quoted 8 lines
Quoted 4 lines
The first patch ("
alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.
Sorry, I should have used "overhead" instead of "regression". When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use of mem_alloc_profiling_key static key. I found a couple of places which were not protected with mem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.
Andrew Morton
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
Quoted 8 lines
Quoted 4 lines
The first patch ("
alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.
Quoted 9 lines
Sorry, I should have used "overhead" instead of "regression". When one sets
CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use ofmem_alloc_profiling_keystatic key. I found a couple of places which were not protected withmem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.
Well, a 50% reduction in a 0.0000000001% overhead ain't much. But I added the final sentence to the changelog.
It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??
Suren Baghdasaryan
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 41 lines · Andrew Morton
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
Quoted 8 lines
Quoted 4 lines
The first patch ("
alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.
Quoted 9 lines
Sorry, I should have used "overhead" instead of "regression". When one sets
CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use ofmem_alloc_profiling_keystatic key. I found a couple of places which were not protected withmem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.
Quoted 1 line
Well, a 50% reduction in a 0.0000000001% overhead ain't much.
I wish the overhead was that low :)
I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%
Quoted 5 lines
But I
added the final sentence to the changelog.It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??
Suren Baghdasaryan
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 48 lines · Suren Baghdasaryan
Quoted 41 lines · Andrew Morton
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
Quoted 8 lines
Quoted 4 lines
The first patch ("
alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.
Quoted 9 lines
Sorry, I should have used "overhead" instead of "regression". When one sets
CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use ofmem_alloc_profiling_keystatic key. I found a couple of places which were not protected withmem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.Quoted 1 line
Well, a 50% reduction in a 0.0000000001% overhead ain't much.
I wish the overhead was that low :)
Quoted 7 lines
I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%
Note, this is an allocation microbenchmark doing allocations in a tight loop. Not a really realistic scenario and useful only to make performance comparisons.
Quoted 7 lines
Quoted 7 lines
But I
added the final sentence to the changelog.It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??
Suren Baghdasaryan
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
In reply to Suren BaghdasaryanQuoted 48 lines · Suren Baghdasaryan
Quoted 41 lines · Andrew Morton
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")Signed-off-by Suren Baghdasaryan <[email protected]>Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.
Quoted 8 lines
Quoted 4 lines
The first patch ("
alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.
Quoted 9 lines
Sorry, I should have used "overhead" instead of "regression". When one sets
CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use ofmem_alloc_profiling_keystatic key. I found a couple of places which were not protected withmem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.Quoted 1 line
Well, a 50% reduction in a 0.0000000001% overhead ain't much.
I wish the overhead was that low :)
Quoted 7 lines
I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%
Hi Andrew,
I just noticed that you added the above results to the description of this patch in mm-unstable: 366507569511 ("alloc_tag: skip pgalloc_tag_swap if profiling is disabled") but this improvement is mostly caused the the other patch in this series: 80aded2b9492 ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled"). If this is not too much trouble, could you please move it into the description of the latter patch?
Thanks,
Suren.
Quoted 7 lines
Quoted 7 lines
But I
added the final sentence to the changelog.It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??
Andrew Morton
maintainerRe: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled
Quoted 12 lines · On Tue, 14 Jan 2025 08:38:37 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 11 lines
Quoted 1 line
Well, a 50% reduction in a 0.0000000001% overhead ain't much.
I wish the overhead was that low :)
I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%
Quoted 8 lines
Hi Andrew,
I just noticed that you added the above results to the description of this patch in mm-unstable: 366507569511 ("alloc_tag: skippgalloc_tag_swapif profiling is disabled") but this improvement is mostly caused the the other patch in this series:80aded2b9492("alloc_tag: avoidcurrent->alloc_tagmanipulations when profiling is disabled"). If this is not too much trouble, could you please move it into the description of the latter patch?
No probs, done, thanks.
Quoted 3 lines · Suren Baghdasaryan
When memory allocation profiling is disabled there is no need to update
current->alloc_tagand these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.
I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?
Quoted 49 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")Signed-off-by Suren Baghdasaryan <[email protected]>--- include/linux/alloc_tag.h | 11 ++++++++--- lib/alloc_tag.c | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-)include/linux/alloc_tag.h +8
-3@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}#define alloc_hooks_tag (_tag, _do_alloc) \ ({ \ - struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag); \ - typeof(_do_alloc) _res = _do_alloc; \ - alloc_tag_restore(_tag, _old); \ + typeof(_do_alloc) _res; \ + if (mem_alloc_profiling_enabled()) { \ + struct alloc_tag * __maybe_unused _old; \ + _old = alloc_tag_save(_tag); \ + _res = _do_alloc; \ + alloc_tag_restore(_tag, _old); \ + } else \ + _res = _do_alloc; \ _res; \ })lib/alloc_tag.c +2
-0@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);DEFINE_STATIC_KEY_MAYBE (CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT , mem_alloc_profiling_key); +EXPORT_SYMBOL (mem_alloc_profiling_key); +DEFINE_STATIC_KEY_FALSE (mem_profiling_compressed);struct alloc_tag_kernel_section kernel_tags = {NULL ,0 };base-commit:
431614f1580a03c1a653340c55ea76bd12a9403fSignature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Suren Baghdasaryan
maintainerQuoted 9 lines · Kent Overstreet
Quoted 3 lines · Suren Baghdasaryan
When memory allocation profiling is disabled there is no need to update
current->alloc_tagand these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?
Hmm but a call
Quoted 50 lines
Quoted 50 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")Signed-off-by Suren Baghdasaryan <[email protected]>--- include/linux/alloc_tag.h | 11 ++++++++--- lib/alloc_tag.c | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-)include/linux/alloc_tag.h +8
-3@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}#define alloc_hooks_tag (_tag, _do_alloc) \ ({ \ - struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag); \ - typeof(_do_alloc) _res = _do_alloc; \ - alloc_tag_restore(_tag, _old); \ + typeof(_do_alloc) _res; \ + if (mem_alloc_profiling_enabled()) { \ + struct alloc_tag * __maybe_unused _old; \ + _old = alloc_tag_save(_tag); \ + _res = _do_alloc; \ + alloc_tag_restore(_tag, _old); \ + } else \ + _res = _do_alloc; \ _res; \ })lib/alloc_tag.c +2
-0@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);DEFINE_STATIC_KEY_MAYBE (CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT , mem_alloc_profiling_key); +EXPORT_SYMBOL (mem_alloc_profiling_key); +DEFINE_STATIC_KEY_FALSE (mem_profiling_compressed);struct alloc_tag_kernel_section kernel_tags = {NULL ,0 };base-commit:
431614f1580a03c1a653340c55ea76bd12a9403f--
2.47.1.613.gc27f4b7a9f-goog
Suren Baghdasaryan
maintainerQuoted 9 lines · Suren Baghdasaryan
Quoted 6 lines · Kent Overstreet
Quoted 3 lines · Suren Baghdasaryan
When memory allocation profiling is disabled there is no need to update
current->alloc_tagand these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.
Quoted 5 lines
Quoted 3 lines
I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?
Hmm but a call
Sorry, I pressed enter before finishing typing.
Is a call to static_branch_maybe() generate a new static key? I thought mem_alloc_profiling_key would be used everywhere but maybe I'm missing something?
Quoted 51 lines
Quoted 51 lines
Quoted 51 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")Signed-off-by Suren Baghdasaryan <[email protected]>--- include/linux/alloc_tag.h | 11 ++++++++--- lib/alloc_tag.c | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-)include/linux/alloc_tag.h +8
-3@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}#define alloc_hooks_tag (_tag, _do_alloc) \ ({ \ - struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag); \ - typeof(_do_alloc) _res = _do_alloc; \ - alloc_tag_restore(_tag, _old); \ + typeof(_do_alloc) _res; \ + if (mem_alloc_profiling_enabled()) { \ + struct alloc_tag * __maybe_unused _old; \ + _old = alloc_tag_save(_tag); \ + _res = _do_alloc; \ + alloc_tag_restore(_tag, _old); \ + } else \ + _res = _do_alloc; \ _res; \ })lib/alloc_tag.c +2
-0@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);DEFINE_STATIC_KEY_MAYBE (CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT , mem_alloc_profiling_key); +EXPORT_SYMBOL (mem_alloc_profiling_key); +DEFINE_STATIC_KEY_FALSE (mem_profiling_compressed);struct alloc_tag_kernel_section kernel_tags = {NULL ,0 };base-commit:
431614f1580a03c1a653340c55ea76bd12a9403f--
2.47.1.613.gc27f4b7a9f-goog
Suren Baghdasaryan
maintainerQuoted 15 lines · Suren Baghdasaryan
Quoted 13 lines · Suren Baghdasaryan
Quoted 9 lines · Kent Overstreet
Quoted 3 lines · Suren Baghdasaryan
When memory allocation profiling is disabled there is no need to update
current->alloc_tagand these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?
Quoted 7 lines
Quoted 1 line
Hmm but a call
Sorry, I pressed enter before finishing typing.
Is a call to
static_branch_maybe()generate a new static key? I thoughtmem_alloc_profiling_keywould be used everywhere but maybe I'm missing something?
Or maybe you meant the NoOp/Jump generated for a static_branch ?
Quoted 52 lines
Quoted 52 lines
Quoted 52 lines
Quoted 52 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")Signed-off-by Suren Baghdasaryan <[email protected]>--- include/linux/alloc_tag.h | 11 ++++++++--- lib/alloc_tag.c | 2 ++ 2 files changed, 10 insertions(+), 3 deletions(-)include/linux/alloc_tag.h +8
-3@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}#define alloc_hooks_tag (_tag, _do_alloc) \ ({ \ - struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag); \ - typeof(_do_alloc) _res = _do_alloc; \ - alloc_tag_restore(_tag, _old); \ + typeof(_do_alloc) _res; \ + if (mem_alloc_profiling_enabled()) { \ + struct alloc_tag * __maybe_unused _old; \ + _old = alloc_tag_save(_tag); \ + _res = _do_alloc; \ + alloc_tag_restore(_tag, _old); \ + } else \ + _res = _do_alloc; \ _res; \ })lib/alloc_tag.c +2
-0@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);DEFINE_STATIC_KEY_MAYBE (CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT , mem_alloc_profiling_key); +EXPORT_SYMBOL (mem_alloc_profiling_key); +DEFINE_STATIC_KEY_FALSE (mem_profiling_compressed);struct alloc_tag_kernel_section kernel_tags = {NULL ,0 };base-commit:
431614f1580a03c1a653340c55ea76bd12a9403f--
2.47.1.613.gc27f4b7a9f-goog
kernel test robot
build botkernel test robot noticed the following build warnings:
Hi Suren,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 431614f1580a03c1a653340c55ea76bd12a9403f]
url: https://github.com/intel-lab-lkp/linux/commits/Suren-Baghdasaryan/alloc_tag-skip-pgalloc_tag_swap-if-profiling-is-disabled/20241227-051744 base: 431614f1580a03c1a653340c55ea76bd12a9403f patch link: https://lore.kernel.org/r/20241226211639.1357704-1-surenb%40google.com patch subject: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled config: x86_64-randconfig-122-20241227 (https://download.01.org/0day-ci/archive/20241227/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241227/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
sparse warnings: (new ones prefixed by >>)
net/ipv6/seg6.c:252:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct in6_addr *t_old @@ got struct in6_addr [noderef] __rcu *tun_src @@
net/ipv6/seg6.c:252:15: sparse: expected struct in6_addr *t_old
net/ipv6/seg6.c:252:15: sparse: got struct in6_addr [noderef] __rcu *tun_src
>> net/ipv6/seg6.c:430:24: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct in6_addr [noderef] __rcu *tun_src @@ got void *[assigned] _res @@
net/ipv6/seg6.c:430:24: sparse: expected struct in6_addr [noderef] __rcu *tun_src
net/ipv6/seg6.c:430:24: sparse: got void *[assigned] _res
vim +430 net/ipv6/seg6.c
4f4853dc1c9c19 David Lebrun 2016-11-08 419
915d7e5e5930b4 David Lebrun 2016-11-08 420 static int __net_init seg6_net_init(struct net *net)
915d7e5e5930b4 David Lebrun 2016-11-08 421 {
915d7e5e5930b4 David Lebrun 2016-11-08 422 struct seg6_pernet_data *sdata;
915d7e5e5930b4 David Lebrun 2016-11-08 423
915d7e5e5930b4 David Lebrun 2016-11-08 424 sdata = kzalloc(sizeof(*sdata), GFP_KERNEL);
915d7e5e5930b4 David Lebrun 2016-11-08 425 if (!sdata)
915d7e5e5930b4 David Lebrun 2016-11-08 426 return -ENOMEM;
915d7e5e5930b4 David Lebrun 2016-11-08 427
915d7e5e5930b4 David Lebrun 2016-11-08 428 mutex_init(&sdata->lock);
915d7e5e5930b4 David Lebrun 2016-11-08 429
915d7e5e5930b4 David Lebrun 2016-11-08 @430 sdata->tun_src = kzalloc(sizeof(*sdata->tun_src), GFP_KERNEL);
915d7e5e5930b4 David Lebrun 2016-11-08 431 if (!sdata->tun_src) {
915d7e5e5930b4 David Lebrun 2016-11-08 432 kfree(sdata);
915d7e5e5930b4 David Lebrun 2016-11-08 433 return -ENOMEM;
915d7e5e5930b4 David Lebrun 2016-11-08 434 }
915d7e5e5930b4 David Lebrun 2016-11-08 435
915d7e5e5930b4 David Lebrun 2016-11-08 436 net->ipv6.seg6_data = sdata;
915d7e5e5930b4 David Lebrun 2016-11-08 437
f04ed7d277e842 MichelleJin 2021-09-27 438 if (seg6_hmac_net_init(net)) {
f04ed7d277e842 MichelleJin 2021-09-27 439 kfree(rcu_dereference_raw(sdata->tun_src));
23b08260481ca5 MichelleJin 2021-10-02 440 kfree(sdata);
f04ed7d277e842 MichelleJin 2021-09-27 441 return -ENOMEM;
acaea0d5a63406 Zhang Mingyu 2021-11-03 442 }
4f4853dc1c9c19 David Lebrun 2016-11-08 443
915d7e5e5930b4 David Lebrun 2016-11-08 444 return 0;
915d7e5e5930b4 David Lebrun 2016-11-08 445 }
915d7e5e5930b4 David Lebrun 2016-11-08 446 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botkernel test robot noticed the following build warnings:
Hi Suren,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 431614f1580a03c1a653340c55ea76bd12a9403f]
url: https://github.com/intel-lab-lkp/linux/commits/Suren-Baghdasaryan/alloc_tag-skip-pgalloc_tag_swap-if-profiling-is-disabled/20241227-051744 base: 431614f1580a03c1a653340c55ea76bd12a9403f patch link: https://lore.kernel.org/r/20241226211639.1357704-1-surenb%40google.com patch subject: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled config: i386-randconfig-061-20241227 (https://download.01.org/0day-ci/archive/20241227/[email protected]/config) compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241227/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
sparse warnings: (new ones prefixed by >>)
kernel/power/energy_model.c:168:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_table [noderef] __rcu *table @@ got struct em_perf_table * @@
kernel/power/energy_model.c:168:15: sparse: expected struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:168:15: sparse: got struct em_perf_table *
kernel/power/energy_model.c:169:15: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const *objp @@ got struct em_perf_table [noderef] __rcu *table @@
kernel/power/energy_model.c:169:15: sparse: expected void const *objp
kernel/power/energy_model.c:169:15: sparse: got struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:177:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_table [noderef] __rcu *table @@ got struct em_perf_table * @@
kernel/power/energy_model.c:177:15: sparse: expected struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:177:15: sparse: got struct em_perf_table *
kernel/power/energy_model.c:179:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct callback_head *head @@ got struct callback_head [noderef] __rcu * @@
kernel/power/energy_model.c:179:19: sparse: expected struct callback_head *head
kernel/power/energy_model.c:179:19: sparse: got struct callback_head [noderef] __rcu *
kernel/power/energy_model.c:190:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct kref *kref @@ got struct kref [noderef] __rcu * @@
kernel/power/energy_model.c:190:19: sparse: expected struct kref *kref
kernel/power/energy_model.c:190:19: sparse: got struct kref [noderef] __rcu *
>> kernel/power/energy_model.c:208:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_table [noderef] __rcu *table @@ got void *[assigned] _res @@
kernel/power/energy_model.c:208:15: sparse: expected struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:208:15: sparse: got void *[assigned] _res
kernel/power/energy_model.c:212:20: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct kref *kref @@ got struct kref [noderef] __rcu * @@
kernel/power/energy_model.c:212:20: sparse: expected struct kref *kref
kernel/power/energy_model.c:212:20: sparse: got struct kref [noderef] __rcu *
kernel/power/energy_model.c:328:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct kref *kref @@ got struct kref [noderef] __rcu * @@
kernel/power/energy_model.c:328:19: sparse: expected struct kref *kref
kernel/power/energy_model.c:328:19: sparse: got struct kref [noderef] __rcu *
kernel/power/energy_model.c:333:45: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:333:45: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:333:45: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:433:45: sparse: sparse: incorrect type in argument 3 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:433:45: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:433:45: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:450:15: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const *objp @@ got struct em_perf_table [noderef] __rcu *[assigned] em_table @@
kernel/power/energy_model.c:450:15: sparse: expected void const *objp
kernel/power/energy_model.c:450:15: sparse: got struct em_perf_table [noderef] __rcu *[assigned] em_table
kernel/power/energy_model.c:634:55: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:634:55: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:634:55: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:689:16: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_state *new_ps @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:689:16: sparse: expected struct em_perf_state *new_ps
kernel/power/energy_model.c:689:16: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:707:37: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:707:37: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:707:37: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:742:38: sparse: sparse: incorrect type in argument 3 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:742:38: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:742:38: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:846:53: sparse: sparse: dereference of noderef expression kernel/power/energy_model.c:855:32: sparse: sparse: dereference of noderef expression
vim +208 kernel/power/energy_model.c
ffcf9bce7af02a Lukasz Luba 2024-02-08 192
ffcf9bce7af02a Lukasz Luba 2024-02-08 193 /**
ffcf9bce7af02a Lukasz Luba 2024-02-08 194 * em_table_alloc() - Allocate a new EM table
ffcf9bce7af02a Lukasz Luba 2024-02-08 195 * @pd : EM performance domain for which this must be done
ffcf9bce7af02a Lukasz Luba 2024-02-08 196 *
ffcf9bce7af02a Lukasz Luba 2024-02-08 197 * Allocate a new EM table and initialize its kref to indicate that it
ffcf9bce7af02a Lukasz Luba 2024-02-08 198 * has a user.
ffcf9bce7af02a Lukasz Luba 2024-02-08 199 * Returns allocated table or NULL.
ffcf9bce7af02a Lukasz Luba 2024-02-08 200 */
ffcf9bce7af02a Lukasz Luba 2024-02-08 201 struct em_perf_table __rcu *em_table_alloc(struct em_perf_domain *pd)
ca0fc871f16f4b Lukasz Luba 2024-02-08 202 {
ca0fc871f16f4b Lukasz Luba 2024-02-08 203 struct em_perf_table __rcu *table;
ca0fc871f16f4b Lukasz Luba 2024-02-08 204 int table_size;
ca0fc871f16f4b Lukasz Luba 2024-02-08 205
ca0fc871f16f4b Lukasz Luba 2024-02-08 206 table_size = sizeof(struct em_perf_state) * pd->nr_perf_states;
ca0fc871f16f4b Lukasz Luba 2024-02-08 207
ca0fc871f16f4b Lukasz Luba 2024-02-08 @208 table = kzalloc(sizeof(*table) + table_size, GFP_KERNEL);
ffcf9bce7af02a Lukasz Luba 2024-02-08 209 if (!table)
ffcf9bce7af02a Lukasz Luba 2024-02-08 210 return NULL;
ffcf9bce7af02a Lukasz Luba 2024-02-08 211
ffcf9bce7af02a Lukasz Luba 2024-02-08 212 kref_init(&table->kref);
ffcf9bce7af02a Lukasz Luba 2024-02-08 213
ca0fc871f16f4b Lukasz Luba 2024-02-08 214 return table;
ca0fc871f16f4b Lukasz Luba 2024-02-08 215 }
ca0fc871f16f4b Lukasz Luba 2024-02-08 216 Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki