Skip to contentSkip to conversation

binfmt_elf: use check_mul_overflow() for size calc

PATCH In review Pranav Tyagi 3 participants 5 messages Jun 7, 2025 – Jun 11, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 binfmt_elf: use check_mul_overflow() for size calc

Use check_mul_overflow() to safely compute the total size of ELF program headers instead of relying on direct multiplication.

Directly multiplying sizeof(struct elf_phdr) with e_phnum risks integer overflow, especially on 32-bit systems or with malformed ELF binaries crafted to trigger wrap-around. If an overflow occurs, kmalloc() could allocate insufficient memory, potentially leading to out-of-bound accesses, memory corruption or security vulnerabilities.

Using check_mul_overflow() ensures the multiplication is performed safely and detects overflows before memory allocation. This change makes the function more robust when handling untrusted or corrupted binaries.

---
 fs/binfmt_elf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/binfmt_elf.c +4 -1
index a43363d593e5..774e705798b8 100644
@@ -518,7 +518,10 @@ static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex,
 
 	/* Sanity check the number of program headers... */
 	/* ...and their total size. */
-	size = sizeof(struct elf_phdr) * elf_ex->e_phnum;
+	
+	if (check_mul_overflow(sizeof(struct elf_phdr), elf_ex->e_phnum, &size))
+		goto out;
+
 	if (size == 0 || size > 65536 || size > ELF_MIN_ALIGN)
 		goto out;
 
Signature · 1 line
2.49.0
reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 9 lines · Pranav Tyagi

Use check_mul_overflow() to safely compute the total size of ELF program headers instead of relying on direct multiplication.

Directly multiplying sizeof(struct elf_phdr) with e_phnum risks integer overflow, especially on 32-bit systems or with malformed ELF binaries crafted to trigger wrap-around. If an overflow occurs, kmalloc() could allocate insufficient memory, potentially leading to out-of-bound accesses, memory corruption or security vulnerabilities.

Quoted 6 lines

Using check_mul_overflow() ensures the multiplication is performed safely and detects overflows before memory allocation. This change makes the function more robust when handling untrusted or corrupted binaries.

Looks good. Feel free to add:

Honza

Quoted 23 lines
---
 fs/binfmt_elf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/binfmt_elf.c +4 -1
index a43363d593e5..774e705798b8 100644
@@ -518,7 +518,10 @@ static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex,
 
 	/* Sanity check the number of program headers... */
 	/* ...and their total size. */
-	size = sizeof(struct elf_phdr) * elf_ex->e_phnum;
+	
+	if (check_mul_overflow(sizeof(struct elf_phdr), elf_ex->e_phnum, &size))
+		goto out;
+
 	if (size == 0 || size > 65536 || size > ELF_MIN_ALIGN)
 		goto out;
 
Signature · 1 line
2.49.0
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
maintainer
#2 lore raw
In reply to Pranav Tyagi
Quoted 24 lines · Pranav Tyagi

Use check_mul_overflow() to safely compute the total size of ELF program headers instead of relying on direct multiplication.

Directly multiplying sizeof(struct elf_phdr) with e_phnum risks integer overflow, especially on 32-bit systems or with malformed ELF binaries crafted to trigger wrap-around. If an overflow occurs, kmalloc() could allocate insufficient memory, potentially leading to out-of-bound accesses, memory corruption or security vulnerabilities.

Using check_mul_overflow() ensures the multiplication is performed safely and detects overflows before memory allocation. This change makes the function more robust when handling untrusted or corrupted binaries.

---
 fs/binfmt_elf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/binfmt_elf.c +0 -0
index a43363d593e5..774e705798b8 100644

fs/binfmt_elf.c static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex, · line 521

Quoted 4 lines
fs/binfmt_elf.c +0 -1
@@ -518,7 +518,10 @@ static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex,
 
 	/* Sanity check the number of program headers... */
 	/* ...and their total size. */
-	size = sizeof(struct elf_phdr) * elf_ex->e_phnum;

size is unsigned int, which has a maximum value of 4,294,967,295.

elf_ex->e_phnum is a u16 (2 bytes) and will not be changing:

$ pahole -C elf64_hdr */fs/binfmt_elf.o
struct elf64_hdr {
	...
        Elf64_Half                 e_phnum;              /*    56     2 */
	...
$ pahole -C Elf64_Half */fs/binfmt_elf.o
typedef __u16 Elf64_Half;

So it has a maximum value of 65,535.

sizeof(struct elf_phdr) is a fixed value, 56:

$ pahole -C elf64_phdr */fs/binfmt_elf.o
struct elf64_phdr {
	...
        /* size: 56, cachelines: 1, members: 8 */
        /* last cacheline: 56 bytes */
};

So the maximum product of the two is 3,669,960.

It is not possible for this calculation to overflow.

fs/binfmt_elf.c static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex, · line 524

Quoted 4 lines
fs/binfmt_elf.c +4 -0
+	
+	if (check_mul_overflow(sizeof(struct elf_phdr), elf_ex->e_phnum, &size))
+		goto out;
+

You can even see that the entire check would be elided by the compiler:

#include <elf.h>

unsigned int unchecked(Elf64_Ehdr *elf_ex)

{
    unsigned int size;

    size = sizeof(Elf64_Phdr) * elf_ex->e_phnum;

    return size;
}

unsigned int checked(Elf64_Ehdr *elf_ex)

{
    unsigned int size;

    if (__builtin_mul_overflow(sizeof(Elf64_Phdr), elf_ex->e_phnum, &size))
        return 0;

    return size;
}

...produces this assembler, identical for both functions:

unchecked:
movzx eax, WORD PTR [rdi+56]
imul eax, eax, 56
ret
checked:
movzx eax, WORD PTR [rdi+56]
imul eax, eax, 56
ret

https://godbolt.org/z/hTEef8cT9

-Kees

Signature · 1 line
Kees Cook
reviewer
#3 lore raw
Quoted 32 lines · Kees Cook
Quoted 24 lines · Pranav Tyagi

Use check_mul_overflow() to safely compute the total size of ELF program headers instead of relying on direct multiplication.

Directly multiplying sizeof(struct elf_phdr) with e_phnum risks integer overflow, especially on 32-bit systems or with malformed ELF binaries crafted to trigger wrap-around. If an overflow occurs, kmalloc() could allocate insufficient memory, potentially leading to out-of-bound accesses, memory corruption or security vulnerabilities.

Using check_mul_overflow() ensures the multiplication is performed safely and detects overflows before memory allocation. This change makes the function more robust when handling untrusted or corrupted binaries.

---
 fs/binfmt_elf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/binfmt_elf.c +0 -0
index a43363d593e5..774e705798b8 100644

fs/binfmt_elf.c static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex, · line 521

Quoted 4 lines
fs/binfmt_elf.c +0 -1
@@ -518,7 +518,10 @@ static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex,
 
 	/* Sanity check the number of program headers... */
 	/* ...and their total size. */
-	size = sizeof(struct elf_phdr) * elf_ex->e_phnum;

size is unsigned int, which has a maximum value of 4,294,967,295.

Quoted 7 lines

elf_ex->e_phnum is a u16 (2 bytes) and will not be changing:

$ pahole -C elf64_hdr */fs/binfmt_elf.o
struct elf64_hdr {
	...
        Elf64_Half                 e_phnum;              /*    56     2 */
	...

Ah, what confused me was that I somehow thought Elf64_Half is u32 without checking it's definition which clearly shows its actually u16. Thanks for checking it! You're right that the patch is pointless then.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#4 lore raw
Quoted 42 lines · Jan Kara
Quoted 32 lines · Kees Cook
Quoted 24 lines · Pranav Tyagi

Use check_mul_overflow() to safely compute the total size of ELF program headers instead of relying on direct multiplication.

Directly multiplying sizeof(struct elf_phdr) with e_phnum risks integer overflow, especially on 32-bit systems or with malformed ELF binaries crafted to trigger wrap-around. If an overflow occurs, kmalloc() could allocate insufficient memory, potentially leading to out-of-bound accesses, memory corruption or security vulnerabilities.

Using check_mul_overflow() ensures the multiplication is performed safely and detects overflows before memory allocation. This change makes the function more robust when handling untrusted or corrupted binaries.

---
 fs/binfmt_elf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
fs/binfmt_elf.c +0 -0
index a43363d593e5..774e705798b8 100644

fs/binfmt_elf.c static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex, · line 521

Quoted 4 lines
fs/binfmt_elf.c +0 -1
@@ -518,7 +518,10 @@ static struct elf_phdr *load_elf_phdrs(const struct elfhdr *elf_ex,
 
    /* Sanity check the number of program headers... */
    /* ...and their total size. */
-   size = sizeof(struct elf_phdr) * elf_ex->e_phnum;

size is unsigned int, which has a maximum value of 4,294,967,295.

Quoted 7 lines

elf_ex->e_phnum is a u16 (2 bytes) and will not be changing:

$ pahole -C elf64_hdr */fs/binfmt_elf.o
struct elf64_hdr {
      ...
        Elf64_Half                 e_phnum;              /*    56     2 */
      ...
Quoted 8 lines

Ah, what confused me was that I somehow thought Elf64_Half is u32 without checking it's definition which clearly shows its actually u16. Thanks for checking it! You're right that the patch is pointless then.

Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR

Hi,

I understand that the patch is actually pointless. I am still new to kernel dev and learnt a lot from your comments. I will keep this in mind while sending patches in the future.

Regards

Pranav Tyagi