Skip to contentSkip to conversation

slab: Provide full coverage for __alloc_size attribute

PATCH In review Kees Cook 3 participants 8 messages Nov 18, 2022 – Nov 21, 2022

Subsystems (2)
lore
Share & tools
lore
author
#1 lore raw

PATCH v2 1/2 slab: Clean up SLOB vs kmalloc() definition

In reply to Kees Cook

As already done for kmalloc_node(), clean up the #ifdef usage in the definition of kmalloc() so that the SLOB-only version is an entirely separate and much more readable function.

Cc Vlastimil Babka <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Roman Gushchin <[email protected]>
Cc Hyeonggon Yoo <[email protected]>
---
 include/linux/slab.h | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
include/linux/slab.h +12 -4
index 1c670c16c737..9033937c758e 100644
@@ -558,15 +558,15 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align
  *	Try really hard to succeed the allocation but fail
  *	eventually.
  */
+#ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 {
 	if (__builtin_constant_p(size)) {
-#ifndef CONFIG_SLOB
 		unsigned int index;
-#endif
+
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large(size, flags);
-#ifndef CONFIG_SLOB
+
 		index = kmalloc_index(size);
 
 		if (!index)
@@ -575,10 +575,18 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 		return kmalloc_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, size);
-#endif
 	}
 	return __kmalloc(size, flags);
 }
+#else
+static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
+{
+	if (__builtin_constant_p(size) && size > KMALLOC_MAX_CACHE_SIZE)
+		return kmalloc_large(size, flags);
+
+	return __kmalloc(size, flags);
+}
+#endif
 
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node)
Signature · 1 line
2.34.1

Kees Cook

author
#0 lore raw

v2 cover slab: Provide full coverage for __alloc_size attribute

Hi,

These patches work around a deficiency in GCC (>=11) and Clang (<16) where the __alloc_size attribute does not apply to inlines. :(
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503

This manifests as reduced overflow detection coverage for many allocation sites under CONFIG_FORTIFY_SOURCE=y, where the allocation size was not actually being propagated to __builtin_dynamic_object_size(). The problem was in two halves: the trace wrapper (now fixed in -next), and const-0 special-casing (covered here).

Thanks,

-Kees

v2:
- drop trace wrapper refactoring -- handled differently now (vbabka)
- drop kunit/fortify test patch since it depends on other changes
- rebase
v1: https://lore.kernel.org/all/[email protected]

Kees Cook (2):
slab: Clean up SLOB vs kmalloc() definition
slab: Remove special-casing of const 0 size allocations

 include/linux/slab.h | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH v2 2/2 slab: Remove special-casing of const 0 size allocations

Passing a constant-0 size allocation into kmalloc() or kmalloc_node() does not need to be a fast-path operation, so the static return value can be removed entirely. This is in preparation for making sure that all paths through the inlines result in a full extern function call, where __alloc_size() hints will actually be seen[1] by GCC. (A constant return value of 0 means the "0" allocation size won't be propagated by the inline.)

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503

Cc Vlastimil Babka <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Roman Gushchin <[email protected]>
Cc Hyeonggon Yoo <[email protected]>
---
 include/linux/slab.h | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
include/linux/slab.h +2 -10
index 9033937c758e..84be05208418 100644
@@ -561,17 +561,13 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 {
-	if (__builtin_constant_p(size)) {
+	if (__builtin_constant_p(size) && size) {
 		unsigned int index;
 
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large(size, flags);
 
 		index = kmalloc_index(size);
-
-		if (!index)
-			return ZERO_SIZE_PTR;
-
 		return kmalloc_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, size);
@@ -591,17 +587,13 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node)
 {
-	if (__builtin_constant_p(size)) {
+	if (__builtin_constant_p(size) && size) {
 		unsigned int index;
 
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large_node(size, flags, node);
 
 		index = kmalloc_index(size);
-
-		if (!index)
-			return ZERO_SIZE_PTR;
-
 		return kmalloc_node_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, node, size);
Signature · 1 line
2.34.1
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] slab: Clean up SLOB vs kmalloc() definition

In reply to Kees Cook
Quoted 63 lines · Kees Cook

As already done for kmalloc_node(), clean up the #ifdef usage in the definition of kmalloc() so that the SLOB-only version is an entirely separate and much more readable function.

Cc Vlastimil Babka <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Roman Gushchin <[email protected]>
Cc Hyeonggon Yoo <[email protected]>
---
 include/linux/slab.h | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)
include/linux/slab.h +12 -4
index 1c670c16c737..9033937c758e 100644
@@ -558,15 +558,15 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align
  *	Try really hard to succeed the allocation but fail
  *	eventually.
  */
+#ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 {
 	if (__builtin_constant_p(size)) {
-#ifndef CONFIG_SLOB
 		unsigned int index;
-#endif
+
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large(size, flags);
-#ifndef CONFIG_SLOB
+
 		index = kmalloc_index(size);
 
 		if (!index)
@@ -575,10 +575,18 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 		return kmalloc_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, size);
-#endif
 	}
 	return __kmalloc(size, flags);
 }
+#else
+static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
+{
+	if (__builtin_constant_p(size) && size > KMALLOC_MAX_CACHE_SIZE)
+		return kmalloc_large(size, flags);
+
+	return __kmalloc(size, flags);
+}
+#endif
 
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node)
Signature · 1 line
2.34.1
Signature · 2 lines
Thanks,
Hyeonggon
#4 lore raw

Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations

In reply to Kees Cook
Quoted 4 lines · Kees Cook

Passing a constant-0 size allocation into kmalloc() or kmalloc_node() does not need to be a fast-path operation, so the static return value can be removed entirely. This is in preparation for making sure that all paths through the inlines result in a full extern function call,

So with the kmalloc_trace() already solved, we could now say it's not "in preparation", but simply "makes sure", right? I can correct that while picking this patch.

Quoted 62 lines

where __alloc_size() hints will actually be seen[1] by GCC. (A constant return value of 0 means the "0" allocation size won't be propagated by the inline.)

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503

Cc Vlastimil Babka <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Roman Gushchin <[email protected]>
Cc Hyeonggon Yoo <[email protected]>
---
 include/linux/slab.h | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
include/linux/slab.h +2 -10
index 9033937c758e..84be05208418 100644
@@ -561,17 +561,13 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 {
-	if (__builtin_constant_p(size)) {
+	if (__builtin_constant_p(size) && size) {
 		unsigned int index;
 
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large(size, flags);
 
 		index = kmalloc_index(size);
-
-		if (!index)
-			return ZERO_SIZE_PTR;
-
 		return kmalloc_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, size);
@@ -591,17 +587,13 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node)
 {
-	if (__builtin_constant_p(size)) {
+	if (__builtin_constant_p(size) && size) {
 		unsigned int index;
 
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large_node(size, flags, node);
 
 		index = kmalloc_index(size);
-
-		if (!index)
-			return ZERO_SIZE_PTR;
-
 		return kmalloc_node_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, node, size);
Reviewed-by on v2 2/2
#7 lore raw

Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations

In reply to Kees Cook
Quoted 68 lines · Kees Cook

Passing a constant-0 size allocation into kmalloc() or kmalloc_node() does not need to be a fast-path operation, so the static return value can be removed entirely. This is in preparation for making sure that all paths through the inlines result in a full extern function call, where __alloc_size() hints will actually be seen[1] by GCC. (A constant return value of 0 means the "0" allocation size won't be propagated by the inline.)

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503

Cc Vlastimil Babka <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Roman Gushchin <[email protected]>
Cc Hyeonggon Yoo <[email protected]>
---
 include/linux/slab.h | 12 ++----------
 1 file changed, 2 insertions(+), 10 deletions(-)
include/linux/slab.h +2 -10
index 9033937c758e..84be05208418 100644
@@ -561,17 +561,13 @@ void *kmalloc_large_node(size_t size, gfp_t flags, int node) __assume_page_align
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 {
-	if (__builtin_constant_p(size)) {
+	if (__builtin_constant_p(size) && size) {
 		unsigned int index;
 
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large(size, flags);
 
 		index = kmalloc_index(size);
-
-		if (!index)
-			return ZERO_SIZE_PTR;
-
 		return kmalloc_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, size);
@@ -591,17 +587,13 @@ static __always_inline __alloc_size(1) void *kmalloc(size_t size, gfp_t flags)
 #ifndef CONFIG_SLOB
 static __always_inline __alloc_size(1) void *kmalloc_node(size_t size, gfp_t flags, int node)
 {
-	if (__builtin_constant_p(size)) {
+	if (__builtin_constant_p(size) && size) {
 		unsigned int index;
 
 		if (size > KMALLOC_MAX_CACHE_SIZE)
 			return kmalloc_large_node(size, flags, node);
 
 		index = kmalloc_index(size);
-
-		if (!index)
-			return ZERO_SIZE_PTR;
-
 		return kmalloc_node_trace(
 				kmalloc_caches[kmalloc_type(flags)][index],
 				flags, node, size);
Signature · 1 line
2.34.1
Signature · 2 lines
Thanks,
Hyeonggon
author
#5 lore raw

Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations

In reply to Vlastimil Babka
Quoted 9 lines · Vlastimil Babka
Quoted 4 lines · Kees Cook

Passing a constant-0 size allocation into kmalloc() or kmalloc_node() does not need to be a fast-path operation, so the static return value can be removed entirely. This is in preparation for making sure that all paths through the inlines result in a full extern function call,

So with the kmalloc_trace() already solved, we could now say it's not "in preparation", but simply "makes sure", right? I can correct that while picking this patch.

Yeah, good point. I missed this when updating the commit logs. Thanks!

Signature · 1 line
Kees Cook
#6 lore raw

Re: [PATCH v2 2/2] slab: Remove special-casing of const 0 size allocations

Quoted 7 lines · Kees Cook
Quoted 6 lines · Vlastimil Babka
Quoted 4 lines · Kees Cook

Passing a constant-0 size allocation into kmalloc() or kmalloc_node() does not need to be a fast-path operation, so the static return value can be removed entirely. This is in preparation for making sure that all paths through the inlines result in a full extern function call,

Quoted 5 lines
Quoted 3 lines

So with the kmalloc_trace() already solved, we could now say it's not "in preparation", but simply "makes sure", right? I can correct that while picking this patch.

Yeah, good point. I missed this when updating the commit logs. Thanks!

Great, pushed to slab/for-6.2/alloc_size