Skip to contentSkip to conversation

framewarn: expand KASAN_EXTRA exception to KASAN

PATCH In review Jason A. Donenfeld 8 participants 11 messages Sep 21, 2018 – Sep 24, 2018 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 framewarn: expand KASAN_EXTRA exception to KASAN

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -1
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
 	int "Warn for stack frames larger than (needs gcc 4.4)"
 	range 0 8192
-	default 3072 if KASAN_EXTRA
+	default 3072 if KASAN
 	default 2048 if GCC_PLUGIN_LATENT_ENTROPY
 	default 1280 if (!64BIT && PARISC)
 	default 1024 if (!64BIT && !PARISC)
Signature · 1 line
2.19.0
#1 lore raw

This patch seems reasonable, but you emailed the wrong people :)

Quoted 26 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -2
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)
--

2.19.0

Acked-by
#2 lore raw
Quoted 13 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 10 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 13 lines

lib/Kconfig.debug config ENABLE_MUST_CHECK · line 228

Quoted 13 lines
lib/Kconfig.debug +1 -1
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)

Looks fine to me.
KASAN always increases stack size too for exactly that reason, so it's reasonable to allow larger frames rather than try to massage code every here and there to fit into the normal frame size.

Thanks

#3 lore raw
Quoted 25 lines · Dmitry Vyukov' via kasan-dev
Quoted 24 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 21 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -1
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
Quoted 8 lines
Quoted 2 lines

lib/Kconfig.debug config ENABLE_MUST_CHECK · line 228

Quoted 2 lines
lib/Kconfig.debug +0 -0
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)

Looks fine to me.
KASAN always increases stack size too for exactly that reason, so it's reasonable to allow larger frames rather than try to massage code every here and there to fit into the normal frame size.

Do we need KASAN_EXTRA then at all?

#4 lore raw
In reply to Andy Lutomirski
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

Quoted 21 lines
Quoted 21 lines
---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -2
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)
--

2.19.0

#5 lore raw
Quoted 10 lines · Andrey Ryabinin
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

Quoted 5 lines

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

+Nick who is also hitting these warnings on clang/arm64 build. As far as I understand the situation there is much worse.

I would be good to understand/fix the worst offenders. But the stack size increase with KASAN is a real, inherent thing. So if we live very close the edge, we can get people using different compilers and/or versions of compilers constantly breaking each other. And clang hits this warnings in lots of places today just because the current code was tailored to gcc over long period, i.e. allowing more locals where gcc happened to handle that better and having fewer locals where gcc happened to handle it worse. But for another compiler all these assumptions are significantly perturbed.

Nick, do you know what frame size limit eliminates the bulk of warnings on clang? Is 3072 a reasonable limit allowing to fix the remaining outliners?

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -2
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)
--

2.19.0

#6 lore raw
Quoted 32 lines · Dmitry Vyukov
Quoted 10 lines · Andrey Ryabinin
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

Quoted 5 lines

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

+Nick who is also hitting these warnings on clang/arm64 build. As far as I understand the situation there is much worse.

I would be good to understand/fix the worst offenders. But the stack size increase with KASAN is a real, inherent thing. So if we live very close the edge, we can get people using different compilers and/or versions of compilers constantly breaking each other. And clang hits this warnings in lots of places today just because the current code was tailored to gcc over long period, i.e. allowing more locals where gcc happened to handle that better and having fewer locals where gcc happened to handle it worse. But for another compiler all these assumptions are significantly perturbed.

Quoted 4 lines

Nick, do you know what frame size limit eliminates the bulk of warnings on clang? Is 3072 a reasonable limit allowing to fix the remaining outliners?

Hi Dmitry,

I know I'm not Nick and I hope I am not butting in but I've been following this thread due to these warnings cropping up in Clang. We've been tracking them on GitHub and judging from the values there, I would argue that 3072 is a good starting value.

Cheers,
Nathan

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -2
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)
--

2.19.0

#7 lore raw
In reply to Dmitry Vyukov
Quoted 33 lines · Dmitry Vyukov
Quoted 10 lines · Andrey Ryabinin
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

Quoted 5 lines

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

+Nick who is also hitting these warnings on clang/arm64 build. As far as I understand the situation there is much worse.

I would be good to understand/fix the worst offenders. But the stack size increase with KASAN is a real, inherent thing. So if we live very close the edge, we can get people using different compilers and/or versions of compilers constantly breaking each other. And clang hits this warnings in lots of places today just because the current code was tailored to gcc over long period, i.e. allowing more locals where gcc happened to handle that better and having fewer locals where gcc happened to handle it worse. But for another compiler all these assumptions are significantly perturbed.

Quoted 3 lines

Nick, do you know what frame size limit eliminates the bulk of warnings on clang? Is 3072 a reasonable limit allowing to fix the remaining outliners?

Here's a plot of the distribution from an arm64 clang build with allyesconfig minus LSE_ATOMICS, BIG_ENDIAN, and GCOV:
https://github.com/ClangBuiltLinux/linux/issues/39#issuecomment-423621655

A value of 3072 would ignore 79 out of the 128 instances (62%).

Some values are huge outliers.

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig.debug +1 -2
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)
--

2.19.0

Signature · 2 lines
Thanks,
~Nick Desaulniers
#8 lore raw
Quoted 39 lines · Nick Desaulniers
Quoted 33 lines · Dmitry Vyukov
Quoted 10 lines · Andrey Ryabinin
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

Quoted 5 lines

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

+Nick who is also hitting these warnings on clang/arm64 build. As far as I understand the situation there is much worse.

I would be good to understand/fix the worst offenders. But the stack size increase with KASAN is a real, inherent thing. So if we live very close the edge, we can get people using different compilers and/or versions of compilers constantly breaking each other. And clang hits this warnings in lots of places today just because the current code was tailored to gcc over long period, i.e. allowing more locals where gcc happened to handle that better and having fewer locals where gcc happened to handle it worse. But for another compiler all these assumptions are significantly perturbed.

Quoted 3 lines

Nick, do you know what frame size limit eliminates the bulk of warnings on clang? Is 3072 a reasonable limit allowing to fix the remaining outliners?

Quoted 5 lines

Here's a plot of the distribution from an arm64 clang build with allyesconfig minus LSE_ATOMICS, BIG_ENDIAN, and GCOV:
https://github.com/ClangBuiltLinux/linux/issues/39#issuecomment-423621655

A value of 3072 would ignore 79 out of the 128 instances (62%).

A value of 4614 would ignore 95% of the instances (2 standard deviations of the distribution of current warnings).

Quoted 31 lines

Some values are huge outliers.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
---
 lib/Kconfig.debug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
lib/Kconfig.debug +0 -0
index 4966c4fbe7f7..39078a080e29 100644
@@ -222,7 +222,7 @@ config ENABLE_MUST_CHECK
 config FRAME_WARN
        int "Warn for stack frames larger than (needs gcc 4.4)"
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
lib/Kconfig.debug +1 -2
        range 0 8192
-       default 3072 if KASAN_EXTRA
+       default 3072 if KASAN
        default 2048 if GCC_PLUGIN_LATENT_ENTROPY
        default 1280 if (!64BIT && PARISC)
        default 1024 if (!64BIT && !PARISC)
--

2.19.0

--
Thanks,
~Nick Desaulniers

Signature · 2 lines
Thanks,
~Nick Desaulniers
#9 lore raw
In reply to Dmitry Vyukov
Quoted 33 lines · Dmitry Vyukov
Quoted 10 lines · Andrey Ryabinin
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

Quoted 5 lines

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

+Nick who is also hitting these warnings on clang/arm64 build. As far as I understand the situation there is much worse.

I would be good to understand/fix the worst offenders. But the stack size increase with KASAN is a real, inherent thing. So if we live very close the edge, we can get people using different compilers and/or versions of compilers constantly breaking each other. And clang hits this warnings in lots of places today just because the current code was tailored to gcc over long period, i.e. allowing more locals where gcc happened to handle that better and having fewer locals where gcc happened to handle it worse. But for another compiler all these assumptions are significantly perturbed.

Quoted 3 lines

Nick, do you know what frame size limit eliminates the bulk of warnings on clang? Is 3072 a reasonable limit allowing to fix the remaining outliners?

I do not consider 3072 a reasonable limit at all. For gcc, we managed to fix or work around all the bugs that caused excessive stack usage. In almost all cases there was something seriously wrong with code generation. I added the KASAN_EXTRA option for the one thing that added an inherent significant overhead to the stack usage.

llvm apparently has a similar bug to what we fixed in gcc. I created a reduced test case for one of the file at:
https://bugs.llvm.org/show_bug.cgi?id=38809

Unfortunately, nobody has commented on that so far, but in the meantime I think the best workaround would be to disable asan-stack entirely when building with clang, and moving it to KASAN_EXTRA there, like we did with the scope check on gcc.

Arnd

#10 lore raw
Quoted 44 lines · Arnd Bergmann
Quoted 33 lines · Dmitry Vyukov
Quoted 10 lines · Andrey Ryabinin
Quoted 8 lines · Andy Lutomirski

This patch seems reasonable, but you emailed the wrong people :)

Quoted 5 lines · Jason A. Donenfeld

It turns out that KASAN in general will bloat stack frames in unexpected ways, not just KASAN_EXTRA. So, this patch trivially changes that default to be associated with KASAN instead of KASAN_EXTRA.

Quoted 5 lines

KASAN_EXTRA bloats stack more than just KASAN, that's why the limit is higher than just for KASAN. If want more details, tead the changelog from commit e7c52b84fb18f08ce49b6067ae6285aca79084a8

If anything causes "stack frame > 2048" warning for KASAN we should at least try to fix it, I mean reduce stack usage.

+Nick who is also hitting these warnings on clang/arm64 build. As far as I understand the situation there is much worse.

I would be good to understand/fix the worst offenders. But the stack size increase with KASAN is a real, inherent thing. So if we live very close the edge, we can get people using different compilers and/or versions of compilers constantly breaking each other. And clang hits this warnings in lots of places today just because the current code was tailored to gcc over long period, i.e. allowing more locals where gcc happened to handle that better and having fewer locals where gcc happened to handle it worse. But for another compiler all these assumptions are significantly perturbed.

Quoted 3 lines

Nick, do you know what frame size limit eliminates the bulk of warnings on clang? Is 3072 a reasonable limit allowing to fix the remaining outliners?

I do not consider 3072 a reasonable limit at all. For gcc, we managed to fix or work around all the bugs that caused excessive stack usage. In almost all cases there was something seriously wrong with code generation. I added the KASAN_EXTRA option for the one thing that added an inherent significant overhead to the stack usage.

Quoted 8 lines

llvm apparently has a similar bug to what we fixed in gcc. I created a reduced test case for one of the file at:
https://bugs.llvm.org/show_bug.cgi?id=38809

Unfortunately, nobody has commented on that so far, but in the meantime I think the best workaround would be to disable asan-stack entirely when building with clang, and moving it to KASAN_EXTRA there, like we did with the scope check on gcc.

Good point. I CCed more people on https://bugs.llvm.org/show_bug.cgi?id=38809