Skip to contentSkip to conversation

alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

PATCH Applied Suren Baghdasaryan 4 participants 17 messages Dec 26, 2024 – Jan 15, 2025

Subsystems (4)
lore
Share & tools
lore

Suren Baghdasaryan

author maintainer
#0 lore raw

PATCH 1/2 alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

When memory allocation profiling is disabled there is no need to update current->alloc_tag and these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.

Fixes b951aaff5035 ("mm: enable page allocation tagging")
---
 include/linux/alloc_tag.h | 11 ++++++++---
 lib/alloc_tag.c           |  2 ++
 2 files changed, 10 insertions(+), 3 deletions(-)
include/linux/alloc_tag.h +8 -3
index 0bbbe537c5f9..a946e0203e6d 100644
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
 
 #define alloc_hooks_tag(_tag, _do_alloc)				\
 ({									\
-	struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);	\
-	typeof(_do_alloc) _res = _do_alloc;				\
-	alloc_tag_restore(_tag, _old);					\
+	typeof(_do_alloc) _res;						\
+	if (mem_alloc_profiling_enabled()) {				\
+		struct alloc_tag * __maybe_unused _old;			\
+		_old = alloc_tag_save(_tag);				\
+		_res = _do_alloc;					\
+		alloc_tag_restore(_tag, _old);				\
+	} else								\
+		_res = _do_alloc;					\
 	_res;								\
 })
 
lib/alloc_tag.c +2 -0
index 7dcebf118a3e..4c373f444eb1 100644
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
 
 DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
 			mem_alloc_profiling_key);
+EXPORT_SYMBOL(mem_alloc_profiling_key);
+
 DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
 
 struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };

base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f

Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
author maintainer
#1 lore raw

PATCH 2/2 alloc_tag: skip pgalloc_tag_swap if profiling is disabled

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
---
 lib/alloc_tag.c | 3 +++
 1 file changed, 3 insertions(+)
lib/alloc_tag.c +3 -0
index 4c373f444eb1..4e5d7af3eaa2 100644
@@ -197,6 +197,9 @@ void pgalloc_tag_swap(struct folio *new, struct folio *old)
 	union codetag_ref ref_old, ref_new;
 	struct alloc_tag *tag_old, *tag_new;
 
+	if (!mem_alloc_profiling_enabled())
+		return;
+
 	tag_old = pgalloc_tag_get(&old->page);
 	if (!tag_old)
 		return;
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
maintainer
#2 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

author maintainer
#3 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 6 lines · Andrew Morton
Quoted 3 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Quoted 7 lines
Quoted 3 lines
Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled. The second one I couldn't really measure, so I think it's not as important. Thanks!

Quoted 1 line
maintainer
#4 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 11 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 10 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
Quoted 8 lines
Quoted 3 lines

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.

um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.

author maintainer
#5 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

Quoted 8 lines
Quoted 4 lines

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.

um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.

Sorry, I should have used "overhead" instead of "regression". When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use of mem_alloc_profiling_key static key. I found a couple of places which were not protected with mem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.

maintainer
#6 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

Quoted 8 lines
Quoted 4 lines

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.

um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.

Quoted 9 lines

Sorry, I should have used "overhead" instead of "regression". When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use of mem_alloc_profiling_key static key. I found a couple of places which were not protected with mem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.

Well, a 50% reduction in a 0.0000000001% overhead ain't much. But I added the final sentence to the changelog.

It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??

author maintainer
#7 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 41 lines · Andrew Morton
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

Quoted 8 lines
Quoted 4 lines

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.

um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.

Quoted 9 lines

Sorry, I should have used "overhead" instead of "regression". When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use of mem_alloc_profiling_key static key. I found a couple of places which were not protected with mem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.

Quoted 1 line

Well, a 50% reduction in a 0.0000000001% overhead ain't much.

I wish the overhead was that low :)

I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:

Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%

Quoted 5 lines

But I
added the final sentence to the changelog.

It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??

author maintainer
#8 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 48 lines · Suren Baghdasaryan
Quoted 41 lines · Andrew Morton
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

Quoted 8 lines
Quoted 4 lines

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.

um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.

Quoted 9 lines

Sorry, I should have used "overhead" instead of "regression". When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use of mem_alloc_profiling_key static key. I found a couple of places which were not protected with mem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.

Quoted 1 line

Well, a 50% reduction in a 0.0000000001% overhead ain't much.

I wish the overhead was that low :)

Quoted 7 lines

I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:

Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%

Note, this is an allocation microbenchmark doing allocations in a tight loop. Not a really realistic scenario and useful only to make performance comparisons.

Quoted 7 lines
Quoted 7 lines

But I
added the final sentence to the changelog.

It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??

author maintainer
#9 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

In reply to Suren Baghdasaryan
Quoted 48 lines · Suren Baghdasaryan
Quoted 41 lines · Andrew Morton
Quoted 28 lines · On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 13 lines · Andrew Morton
Quoted 6 lines · On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <[email protected]> wrote:

When memory allocation profiling is disabled, there is no need to swap allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")

Are these changes worth backporting? Some indication of how much difference the patches make would help people understand why we're proposing a backport.

Quoted 8 lines
Quoted 4 lines

The first patch ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled") I think is worth backporting. It eliminates about half of the regression for slab allocations when profiling is disabled.

um, what regression? The changelog makes no mention of this. Please send along a suitable Reported-by: and Closes: and a summary of the benefits so that people can actually see what this patch does, and why.

Quoted 9 lines

Sorry, I should have used "overhead" instead of "regression". When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented and even if profiling is turned off, it still has a small performance cost minimized by the use of mem_alloc_profiling_key static key. I found a couple of places which were not protected with mem_alloc_profiling_key, which means that even when profiling is turned off, the code is still executed. Once I added these checks, the overhead of the mode when memory profiling is enabled but turned off went down by about 50%.

Quoted 1 line

Well, a 50% reduction in a 0.0000000001% overhead ain't much.

I wish the overhead was that low :)

Quoted 7 lines

I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:

Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%

Hi Andrew,
I just noticed that you added the above results to the description of this patch in mm-unstable: 366507569511 ("alloc_tag: skip pgalloc_tag_swap if profiling is disabled") but this improvement is mostly caused the the other patch in this series: 80aded2b9492 ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled"). If this is not too much trouble, could you please move it into the description of the latter patch?
Thanks,
Suren.

Quoted 7 lines
Quoted 7 lines

But I
added the final sentence to the changelog.

It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??

maintainer
#10 lore raw

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

Quoted 12 lines · On Tue, 14 Jan 2025 08:38:37 -0800 Suren Baghdasaryan <[email protected]> wrote:
Quoted 11 lines
Quoted 1 line

Well, a 50% reduction in a 0.0000000001% overhead ain't much.

I wish the overhead was that low :)

I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:

Overhead before fixes Overhead after fixes
slab alloc page alloc slab alloc page alloc
Big 6.21% 5.32% 3.31% 4.93%
Medium 4.51% 5.05% 3.79% 4.39%
Little 7.62% 1.82% 6.68% 1.02%

Quoted 8 lines

Hi Andrew,
I just noticed that you added the above results to the description of this patch in mm-unstable: 366507569511 ("alloc_tag: skip pgalloc_tag_swap if profiling is disabled") but this improvement is mostly caused the the other patch in this series: 80aded2b9492 ("alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled"). If this is not too much trouble, could you please move it into the description of the latter patch?

No probs, done, thanks.

#11 lore raw
In reply to Suren Baghdasaryan
Quoted 3 lines · Suren Baghdasaryan

When memory allocation profiling is disabled there is no need to update current->alloc_tag and these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.

I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?

Quoted 49 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")
---
 include/linux/alloc_tag.h | 11 ++++++++---
 lib/alloc_tag.c           |  2 ++
 2 files changed, 10 insertions(+), 3 deletions(-)
include/linux/alloc_tag.h +8 -3
index 0bbbe537c5f9..a946e0203e6d 100644
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
 
 #define alloc_hooks_tag(_tag, _do_alloc)				\
 ({									\
-	struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);	\
-	typeof(_do_alloc) _res = _do_alloc;				\
-	alloc_tag_restore(_tag, _old);					\
+	typeof(_do_alloc) _res;						\
+	if (mem_alloc_profiling_enabled()) {				\
+		struct alloc_tag * __maybe_unused _old;			\
+		_old = alloc_tag_save(_tag);				\
+		_res = _do_alloc;					\
+		alloc_tag_restore(_tag, _old);				\
+	} else								\
+		_res = _do_alloc;					\
 	_res;								\
 })
 
lib/alloc_tag.c +2 -0
index 7dcebf118a3e..4c373f444eb1 100644
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
 
 DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
 			mem_alloc_profiling_key);
+EXPORT_SYMBOL(mem_alloc_profiling_key);
+
 DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
 
 struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };

base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f

Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
author maintainer
#12 lore raw
Quoted 9 lines · Kent Overstreet
Quoted 3 lines · Suren Baghdasaryan

When memory allocation profiling is disabled there is no need to update current->alloc_tag and these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.

I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?

Hmm but a call

Quoted 50 lines
Quoted 50 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")
---
 include/linux/alloc_tag.h | 11 ++++++++---
 lib/alloc_tag.c           |  2 ++
 2 files changed, 10 insertions(+), 3 deletions(-)
include/linux/alloc_tag.h +8 -3
index 0bbbe537c5f9..a946e0203e6d 100644
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
 
 #define alloc_hooks_tag(_tag, _do_alloc)                             \
 ({                                                                   \
-     struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);  \
-     typeof(_do_alloc) _res = _do_alloc;                             \
-     alloc_tag_restore(_tag, _old);                                  \
+     typeof(_do_alloc) _res;                                         \
+     if (mem_alloc_profiling_enabled()) {                            \
+             struct alloc_tag * __maybe_unused _old;                 \
+             _old = alloc_tag_save(_tag);                            \
+             _res = _do_alloc;                                       \
+             alloc_tag_restore(_tag, _old);                          \
+     } else                                                          \
+             _res = _do_alloc;                                       \
      _res;                                                           \
 })
 
lib/alloc_tag.c +2 -0
index 7dcebf118a3e..4c373f444eb1 100644
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
 
 DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
                      mem_alloc_profiling_key);
+EXPORT_SYMBOL(mem_alloc_profiling_key);
+
 DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
 
 struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };

base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f --
2.47.1.613.gc27f4b7a9f-goog

author maintainer
#13 lore raw
Quoted 9 lines · Suren Baghdasaryan
Quoted 6 lines · Kent Overstreet
Quoted 3 lines · Suren Baghdasaryan

When memory allocation profiling is disabled there is no need to update current->alloc_tag and these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.

Quoted 5 lines
Quoted 3 lines

I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?

Hmm but a call

Sorry, I pressed enter before finishing typing.

Is a call to static_branch_maybe() generate a new static key? I thought mem_alloc_profiling_key would be used everywhere but maybe I'm missing something?

Quoted 51 lines
Quoted 51 lines
Quoted 51 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")
---
 include/linux/alloc_tag.h | 11 ++++++++---
 lib/alloc_tag.c           |  2 ++
 2 files changed, 10 insertions(+), 3 deletions(-)
include/linux/alloc_tag.h +8 -3
index 0bbbe537c5f9..a946e0203e6d 100644
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
 
 #define alloc_hooks_tag(_tag, _do_alloc)                             \
 ({                                                                   \
-     struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);  \
-     typeof(_do_alloc) _res = _do_alloc;                             \
-     alloc_tag_restore(_tag, _old);                                  \
+     typeof(_do_alloc) _res;                                         \
+     if (mem_alloc_profiling_enabled()) {                            \
+             struct alloc_tag * __maybe_unused _old;                 \
+             _old = alloc_tag_save(_tag);                            \
+             _res = _do_alloc;                                       \
+             alloc_tag_restore(_tag, _old);                          \
+     } else                                                          \
+             _res = _do_alloc;                                       \
      _res;                                                           \
 })
 
lib/alloc_tag.c +2 -0
index 7dcebf118a3e..4c373f444eb1 100644
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
 
 DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
                      mem_alloc_profiling_key);
+EXPORT_SYMBOL(mem_alloc_profiling_key);
+
 DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
 
 struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };

base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f --
2.47.1.613.gc27f4b7a9f-goog

author maintainer
#14 lore raw
Quoted 15 lines · Suren Baghdasaryan
Quoted 13 lines · Suren Baghdasaryan
Quoted 9 lines · Kent Overstreet
Quoted 3 lines · Suren Baghdasaryan

When memory allocation profiling is disabled there is no need to update current->alloc_tag and these manipulations add unnecessary overhead. Fix the overhead by skipping these extra updates.

I did it the other way because I was concerned about the overhead of adding a huge number of static keys. But on further thought a static key probably isn't any bigger than an alloc tag, no?

Quoted 7 lines
Quoted 1 line

Hmm but a call

Sorry, I pressed enter before finishing typing.

Is a call to static_branch_maybe() generate a new static key? I thought mem_alloc_profiling_key would be used everywhere but maybe I'm missing something?

Or maybe you meant the NoOp/Jump generated for a static_branch ?

Quoted 52 lines
Quoted 52 lines
Quoted 52 lines
Quoted 52 lines
Fixes b951aaff5035 ("mm: enable page allocation tagging")
---
 include/linux/alloc_tag.h | 11 ++++++++---
 lib/alloc_tag.c           |  2 ++
 2 files changed, 10 insertions(+), 3 deletions(-)
include/linux/alloc_tag.h +8 -3
index 0bbbe537c5f9..a946e0203e6d 100644
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
 
 #define alloc_hooks_tag(_tag, _do_alloc)                             \
 ({                                                                   \
-     struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);  \
-     typeof(_do_alloc) _res = _do_alloc;                             \
-     alloc_tag_restore(_tag, _old);                                  \
+     typeof(_do_alloc) _res;                                         \
+     if (mem_alloc_profiling_enabled()) {                            \
+             struct alloc_tag * __maybe_unused _old;                 \
+             _old = alloc_tag_save(_tag);                            \
+             _res = _do_alloc;                                       \
+             alloc_tag_restore(_tag, _old);                          \
+     } else                                                          \
+             _res = _do_alloc;                                       \
      _res;                                                           \
 })
 
lib/alloc_tag.c +2 -0
index 7dcebf118a3e..4c373f444eb1 100644
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
 
 DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
                      mem_alloc_profiling_key);
+EXPORT_SYMBOL(mem_alloc_profiling_key);
+
 DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
 
 struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };

base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f --
2.47.1.613.gc27f4b7a9f-goog

#15 lore raw
In reply to Suren Baghdasaryan
kernel test robot noticed the following build warnings:

Hi Suren,

kernel test robot noticed the following build warnings:

[auto build test WARNING on 431614f1580a03c1a653340c55ea76bd12a9403f]

url:    https://github.com/intel-lab-lkp/linux/commits/Suren-Baghdasaryan/alloc_tag-skip-pgalloc_tag_swap-if-profiling-is-disabled/20241227-051744
base:   431614f1580a03c1a653340c55ea76bd12a9403f
patch link:    https://lore.kernel.org/r/20241226211639.1357704-1-surenb%40google.com
patch subject: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled
config: x86_64-randconfig-122-20241227 (https://download.01.org/0day-ci/archive/20241227/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241227/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)
net/ipv6/seg6.c:252:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct in6_addr *t_old @@ got struct in6_addr [noderef] __rcu *tun_src @@
net/ipv6/seg6.c:252:15: sparse: expected struct in6_addr *t_old
net/ipv6/seg6.c:252:15: sparse: got struct in6_addr [noderef] __rcu *tun_src

>> net/ipv6/seg6.c:430:24: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct in6_addr [noderef] __rcu *tun_src @@     got void *[assigned] _res @@

net/ipv6/seg6.c:430:24: sparse: expected struct in6_addr [noderef] __rcu *tun_src
net/ipv6/seg6.c:430:24: sparse: got void *[assigned] _res

vim +430 net/ipv6/seg6.c

4f4853dc1c9c19 David Lebrun 2016-11-08  419  
915d7e5e5930b4 David Lebrun 2016-11-08  420  static int __net_init seg6_net_init(struct net *net)
915d7e5e5930b4 David Lebrun 2016-11-08  421  {
915d7e5e5930b4 David Lebrun 2016-11-08  422  	struct seg6_pernet_data *sdata;
915d7e5e5930b4 David Lebrun 2016-11-08  423  
915d7e5e5930b4 David Lebrun 2016-11-08  424  	sdata = kzalloc(sizeof(*sdata), GFP_KERNEL);
915d7e5e5930b4 David Lebrun 2016-11-08  425  	if (!sdata)
915d7e5e5930b4 David Lebrun 2016-11-08  426  		return -ENOMEM;
915d7e5e5930b4 David Lebrun 2016-11-08  427  
915d7e5e5930b4 David Lebrun 2016-11-08  428  	mutex_init(&sdata->lock);
915d7e5e5930b4 David Lebrun 2016-11-08  429  
915d7e5e5930b4 David Lebrun 2016-11-08 @430  	sdata->tun_src = kzalloc(sizeof(*sdata->tun_src), GFP_KERNEL);
915d7e5e5930b4 David Lebrun 2016-11-08  431  	if (!sdata->tun_src) {
915d7e5e5930b4 David Lebrun 2016-11-08  432  		kfree(sdata);
915d7e5e5930b4 David Lebrun 2016-11-08  433  		return -ENOMEM;
915d7e5e5930b4 David Lebrun 2016-11-08  434  	}
915d7e5e5930b4 David Lebrun 2016-11-08  435  
915d7e5e5930b4 David Lebrun 2016-11-08  436  	net->ipv6.seg6_data = sdata;
915d7e5e5930b4 David Lebrun 2016-11-08  437  
f04ed7d277e842 MichelleJin  2021-09-27  438  	if (seg6_hmac_net_init(net)) {
f04ed7d277e842 MichelleJin  2021-09-27  439  		kfree(rcu_dereference_raw(sdata->tun_src));
23b08260481ca5 MichelleJin  2021-10-02  440  		kfree(sdata);
f04ed7d277e842 MichelleJin  2021-09-27  441  		return -ENOMEM;
acaea0d5a63406 Zhang Mingyu 2021-11-03  442  	}
4f4853dc1c9c19 David Lebrun 2016-11-08  443  
915d7e5e5930b4 David Lebrun 2016-11-08  444  	return 0;
915d7e5e5930b4 David Lebrun 2016-11-08  445  }
915d7e5e5930b4 David Lebrun 2016-11-08  446  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#16 lore raw
In reply to Suren Baghdasaryan
kernel test robot noticed the following build warnings:

Hi Suren,

kernel test robot noticed the following build warnings:

[auto build test WARNING on 431614f1580a03c1a653340c55ea76bd12a9403f]

url:    https://github.com/intel-lab-lkp/linux/commits/Suren-Baghdasaryan/alloc_tag-skip-pgalloc_tag_swap-if-profiling-is-disabled/20241227-051744
base:   431614f1580a03c1a653340c55ea76bd12a9403f
patch link:    https://lore.kernel.org/r/20241226211639.1357704-1-surenb%40google.com
patch subject: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled
config: i386-randconfig-061-20241227 (https://download.01.org/0day-ci/archive/20241227/[email protected]/config)
compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241227/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)
kernel/power/energy_model.c:168:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_table [noderef] __rcu *table @@ got struct em_perf_table * @@
kernel/power/energy_model.c:168:15: sparse: expected struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:168:15: sparse: got struct em_perf_table *
kernel/power/energy_model.c:169:15: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const *objp @@ got struct em_perf_table [noderef] __rcu *table @@
kernel/power/energy_model.c:169:15: sparse: expected void const *objp
kernel/power/energy_model.c:169:15: sparse: got struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:177:15: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_table [noderef] __rcu *table @@ got struct em_perf_table * @@
kernel/power/energy_model.c:177:15: sparse: expected struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:177:15: sparse: got struct em_perf_table *
kernel/power/energy_model.c:179:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct callback_head *head @@ got struct callback_head [noderef] __rcu * @@
kernel/power/energy_model.c:179:19: sparse: expected struct callback_head *head
kernel/power/energy_model.c:179:19: sparse: got struct callback_head [noderef] __rcu *
kernel/power/energy_model.c:190:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct kref *kref @@ got struct kref [noderef] __rcu * @@
kernel/power/energy_model.c:190:19: sparse: expected struct kref *kref
kernel/power/energy_model.c:190:19: sparse: got struct kref [noderef] __rcu *

>> kernel/power/energy_model.c:208:15: sparse: sparse: incorrect type in assignment (different address spaces) @@     expected struct em_perf_table [noderef] __rcu *table @@     got void *[assigned] _res @@

kernel/power/energy_model.c:208:15: sparse: expected struct em_perf_table [noderef] __rcu *table
kernel/power/energy_model.c:208:15: sparse: got void *[assigned] _res
kernel/power/energy_model.c:212:20: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct kref *kref @@ got struct kref [noderef] __rcu * @@
kernel/power/energy_model.c:212:20: sparse: expected struct kref *kref
kernel/power/energy_model.c:212:20: sparse: got struct kref [noderef] __rcu *
kernel/power/energy_model.c:328:19: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected struct kref *kref @@ got struct kref [noderef] __rcu * @@
kernel/power/energy_model.c:328:19: sparse: expected struct kref *kref
kernel/power/energy_model.c:328:19: sparse: got struct kref [noderef] __rcu *
kernel/power/energy_model.c:333:45: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:333:45: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:333:45: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:433:45: sparse: sparse: incorrect type in argument 3 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:433:45: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:433:45: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:450:15: sparse: sparse: incorrect type in argument 1 (different address spaces) @@ expected void const *objp @@ got struct em_perf_table [noderef] __rcu *[assigned] em_table @@
kernel/power/energy_model.c:450:15: sparse: expected void const *objp
kernel/power/energy_model.c:450:15: sparse: got struct em_perf_table [noderef] __rcu *[assigned] em_table
kernel/power/energy_model.c:634:55: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:634:55: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:634:55: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:689:16: sparse: sparse: incorrect type in assignment (different address spaces) @@ expected struct em_perf_state *new_ps @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:689:16: sparse: expected struct em_perf_state *new_ps
kernel/power/energy_model.c:689:16: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:707:37: sparse: sparse: incorrect type in argument 2 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:707:37: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:707:37: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:742:38: sparse: sparse: incorrect type in argument 3 (different address spaces) @@ expected struct em_perf_state *table @@ got struct em_perf_state [noderef] __rcu * @@
kernel/power/energy_model.c:742:38: sparse: expected struct em_perf_state *table
kernel/power/energy_model.c:742:38: sparse: got struct em_perf_state [noderef] __rcu *
kernel/power/energy_model.c:846:53: sparse: sparse: dereference of noderef expression kernel/power/energy_model.c:855:32: sparse: sparse: dereference of noderef expression

vim +208 kernel/power/energy_model.c

ffcf9bce7af02a Lukasz Luba 2024-02-08  192  
ffcf9bce7af02a Lukasz Luba 2024-02-08  193  /**
ffcf9bce7af02a Lukasz Luba 2024-02-08  194   * em_table_alloc() - Allocate a new EM table
ffcf9bce7af02a Lukasz Luba 2024-02-08  195   * @pd		: EM performance domain for which this must be done
ffcf9bce7af02a Lukasz Luba 2024-02-08  196   *
ffcf9bce7af02a Lukasz Luba 2024-02-08  197   * Allocate a new EM table and initialize its kref to indicate that it
ffcf9bce7af02a Lukasz Luba 2024-02-08  198   * has a user.
ffcf9bce7af02a Lukasz Luba 2024-02-08  199   * Returns allocated table or NULL.
ffcf9bce7af02a Lukasz Luba 2024-02-08  200   */
ffcf9bce7af02a Lukasz Luba 2024-02-08  201  struct em_perf_table __rcu *em_table_alloc(struct em_perf_domain *pd)
ca0fc871f16f4b Lukasz Luba 2024-02-08  202  {
ca0fc871f16f4b Lukasz Luba 2024-02-08  203  	struct em_perf_table __rcu *table;
ca0fc871f16f4b Lukasz Luba 2024-02-08  204  	int table_size;
ca0fc871f16f4b Lukasz Luba 2024-02-08  205  
ca0fc871f16f4b Lukasz Luba 2024-02-08  206  	table_size = sizeof(struct em_perf_state) * pd->nr_perf_states;
ca0fc871f16f4b Lukasz Luba 2024-02-08  207  
ca0fc871f16f4b Lukasz Luba 2024-02-08 @208  	table = kzalloc(sizeof(*table) + table_size, GFP_KERNEL);
ffcf9bce7af02a Lukasz Luba 2024-02-08  209  	if (!table)
ffcf9bce7af02a Lukasz Luba 2024-02-08  210  		return NULL;
ffcf9bce7af02a Lukasz Luba 2024-02-08  211  
ffcf9bce7af02a Lukasz Luba 2024-02-08  212  	kref_init(&table->kref);
ffcf9bce7af02a Lukasz Luba 2024-02-08  213  
ca0fc871f16f4b Lukasz Luba 2024-02-08  214  	return table;
ca0fc871f16f4b Lukasz Luba 2024-02-08  215  }
ca0fc871f16f4b Lukasz Luba 2024-02-08  216  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki