arm64: remove special treatment for the link order of head.o

Date: Thu, 13 Oct 2022 08:35:00 +0900
From: Masahiro Yamada <[email protected]>
To: Catalin Marinas <[email protected]>, Will Deacon
<[email protected]>, [email protected]
Cc: [email protected], Ard Biesheuvel <[email protected]>, Masahiro
Yamada <[email protected]>, Nicolas Schier <[email protected]>, [email protected]
Subject: [PATCH] arm64: remove special treatment for the link order of
head.o
Message-Id: <[email protected]>
X-Mailer: git-send-email 2.34.1

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -1

index b16326a92c45..f226e45e3b7b 100644
@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

Signature · 1 line

2.34.1

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

[...]

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

Quoted 37 lines · Aurelien Jarno

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

Quoted 3 lines

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

I cannot reproduce this with mainline

With the patch

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752630 621680 39941619 26175f3 vmlinux

With the patch reverted

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752694 621680 39941683 2617633 vmlinux

It would help to compare the resulting vmlinux ELF images from both builds to see where the extra space is being allocated

Hi,

Quoted 50 lines · Ard Biesheuvel

Quoted 37 lines · Aurelien Jarno

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

Quoted 3 lines

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

I cannot reproduce this with mainline

With the patch

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752630 621680 39941619 26175f3 vmlinux

Quoted 5 lines

With the patch reverted

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752694 621680 39941683 2617633 vmlinux

I have tried with the current mainline, this is what I get, using GCC 12.2.0 and binutils 2.40:

text data bss dec hex filename

32531655        8192996  621968 41346619        276e63b vmlinux.orig
25170610        8192996  621968 33985574        2069426 vmlinux.revert

Quoted 2 lines

It would help to compare the resulting vmlinux ELF images from both builds to see where the extra space is being allocated

At a first glance, it seems the extra space is allocated in the BTF section. I have uploaded the resulting files as well as the config file I used there:
https://temp.aurel32.net/linux-arm64-size-head.o.tar.gz

(cc BTF list and maintainer)

Quoted 67 lines · Aurelien Jarno

Hi,

Quoted 50 lines · Ard Biesheuvel

Quoted 37 lines · Aurelien Jarno

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

Quoted 3 lines

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

I cannot reproduce this with mainline

With the patch

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752630 621680 39941619 26175f3 vmlinux

Quoted 5 lines

With the patch reverted

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752694 621680 39941683 2617633 vmlinux

I have tried with the current mainline, this is what I get, using GCC 12.2.0 and binutils 2.40:

text data bss dec hex filename

32531655        8192996  621968 41346619        276e63b vmlinux.orig
25170610        8192996  621968 33985574        2069426 vmlinux.revert

Quoted 8 lines

Quoted 2 lines

It would help to compare the resulting vmlinux ELF images from both builds to see where the extra space is being allocated

At a first glance, it seems the extra space is allocated in the BTF section. I have uploaded the resulting files as well as the config file I used there:
https://temp.aurel32.net/linux-arm64-size-head.o.tar.gz

Indeed. So we go from

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       00000000005093d6  0000000000000000   A       0     0     1

to

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       0000000000c0e5eb  0000000000000000   A       0     0     1

i.e, from 5 MiB to 12+ MiB of BTF metadata.

To me, it is not clear at all how one would be related to the other, so it will leave it to the Kbuild and BTF experts to chew on this one.

Quoted 87 lines · Ard Biesheuvel

(cc BTF list and maintainer)

Quoted 67 lines · Aurelien Jarno

Hi,

Quoted 50 lines · Ard Biesheuvel

Quoted 37 lines · Aurelien Jarno

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

Quoted 3 lines

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

I cannot reproduce this with mainline

With the patch

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752630 621680 39941619 26175f3 vmlinux

Quoted 5 lines

With the patch reverted

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752694 621680 39941683 2617633 vmlinux

I have tried with the current mainline, this is what I get, using GCC 12.2.0 and binutils 2.40:

text data bss dec hex filename

32531655        8192996  621968 41346619        276e63b vmlinux.orig
25170610        8192996  621968 33985574        2069426 vmlinux.revert

Quoted 8 lines

Quoted 2 lines

It would help to compare the resulting vmlinux ELF images from both builds to see where the extra space is being allocated

At a first glance, it seems the extra space is allocated in the BTF section. I have uploaded the resulting files as well as the config file I used there:
https://temp.aurel32.net/linux-arm64-size-head.o.tar.gz

Indeed. So we go from

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       00000000005093d6  0000000000000000   A       0     0     1

to

Quoted 7 lines

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       0000000000c0e5eb  0000000000000000   A       0     0     1

i.e, from 5 MiB to 12+ MiB of BTF metadata.

To me, it is not clear at all how one would be related to the other, so it will leave it to the Kbuild and BTF experts to chew on this one.

That's a huge increase.
It's not just that commit responsible, but the whole series ?
https://lore.kernel.org/lkml/[email protected]/
I'm guessing "Link vmlinux and modules in parallel" is related. I'm not sure what "parallel link" means. Running 'ar' in parallel? I cannot read makefile syntax, so no idea.

Jiri, Andrii, Alan, please take a look.

(cc BTF list and maintainer)

Quoted 67 lines · Aurelien Jarno

Hi,

Quoted 50 lines · Ard Biesheuvel

Quoted 37 lines · Aurelien Jarno

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

Quoted 3 lines

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

I cannot reproduce this with mainline

With the patch

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752630 621680 39941619 26175f3 vmlinux

Quoted 5 lines

With the patch reverted

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752694 621680 39941683 2617633 vmlinux

I have tried with the current mainline, this is what I get, using GCC 12.2.0 and binutils 2.40:

text data bss dec hex filename

32531655        8192996  621968 41346619        276e63b vmlinux.orig
25170610        8192996  621968 33985574        2069426 vmlinux.revert

Quoted 8 lines

Quoted 2 lines

It would help to compare the resulting vmlinux ELF images from both builds to see where the extra space is being allocated

At a first glance, it seems the extra space is allocated in the BTF section. I have uploaded the resulting files as well as the config file I used there:
https://temp.aurel32.net/linux-arm64-size-head.o.tar.gz

Indeed. So we go from

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       00000000005093d6  0000000000000000   A       0     0     1

to

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       0000000000c0e5eb  0000000000000000   A       0     0     1

i.e, from 5 MiB to 12+ MiB of BTF metadata.

To me, it is not clear at all how one would be related to the other, so it will leave it to the Kbuild and BTF experts to chew on this one.

Quoted 87 lines · Ard Biesheuvel

(cc BTF list and maintainer)

Quoted 67 lines · Aurelien Jarno

Hi,

Quoted 50 lines · Ard Biesheuvel

Quoted 37 lines · Aurelien Jarno

Hi,

Quoted 21 lines · Masahiro Yamada

In the previous discussion (see the Link tag), Ard pointed out that arm/arm64/kernel/head.o does not need any special treatment - the only piece that must appear right at the start of the binary image is the image header which is emitted into .head.text.

The linker script does the right thing to do. The build system does not need to manipulate the link order of head.o.

Link https://lore.kernel.org/lkml/CAMj1kXH77Ja8bSsq2Qj8Ck9iSZKw=1F8Uy-uAWGVDm4-CG=EuA@mail.gmail.com/ open here

Suggested-by Ard Biesheuvel <[email protected]>

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/head-object-list.txt | 1 -
 1 file changed, 1 deletion(-)

scripts/head-object-list.txt +0 -0

index b16326a92c45..f226e45e3b7b 100644

scripts/head-object-list.txt arch/alpha/kernel/head.o · line 20

Quoted 7 lines

scripts/head-object-list.txt +0 -1

@@ -15,7 +15,6 @@ arch/alpha/kernel/head.o
 arch/arc/kernel/head.o
 arch/arm/kernel/head-nommu.o
 arch/arm/kernel/head.o
-arch/arm64/kernel/head.o
 arch/csky/kernel/head.o
 arch/hexagon/kernel/head.o
 arch/ia64/kernel/head.o

This patch causes a significant increase of the arch/arm64/boot/Image size. For instance the generic arm64 Debian kernel went from 31 to 39 MB after this patch has been applied to the 6.1 stable tree.

Quoted 3 lines

In turn this causes issues with some bootloaders, for instance U-Boot on a Raspberry Pi limits the kernel size to 36 MB.

I cannot reproduce this with mainline

With the patch

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752630 621680 39941619 26175f3 vmlinux

Quoted 5 lines

With the patch reverted

$ size vmlinux
   text    data     bss     dec     hex filename
24567309 14752694 621680 39941683 2617633 vmlinux

I have tried with the current mainline, this is what I get, using GCC 12.2.0 and binutils 2.40:

text data bss dec hex filename

32531655        8192996  621968 41346619        276e63b vmlinux.orig
25170610        8192996  621968 33985574        2069426 vmlinux.revert

Quoted 8 lines

Quoted 2 lines

It would help to compare the resulting vmlinux ELF images from both builds to see where the extra space is being allocated

At a first glance, it seems the extra space is allocated in the BTF section. I have uploaded the resulting files as well as the config file I used there:
https://temp.aurel32.net/linux-arm64-size-head.o.tar.gz

Indeed. So we go from

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       00000000005093d6  0000000000000000   A       0     0     1

to

Quoted 7 lines

[15] .BTF PROGBITS ffff8000091d1ff4 011e1ff4

       0000000000c0e5eb  0000000000000000   A       0     0     1

i.e, from 5 MiB to 12+ MiB of BTF metadata.

To me, it is not clear at all how one would be related to the other, so it will leave it to the Kbuild and BTF experts to chew on this one.

Strange.

I used the .config file Aurelien provided, but
I still cannot reproduce this issue.

The vmlinux size is small
as-is in the current mainline.

[mainline]

masahiro@zoe:~/ref/linux(master)$ git log --oneline -1
65aca32efdcb (HEAD -> master, origin/master, origin/HEAD) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561282 8186912 622032 33370226 1fd3072 vmlinux
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       000000000048209c  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427a4  016527a4
       0000000000000a1c  0000000000000000   A       0     0     1

[mainline + revert 994b7ac]

masahiro@zoe:~/ref/linux2(testing)$ git log --oneline -2
856c80dd789c (HEAD -> testing) Revert "arm64: remove special treatment
for the link order of head.o"
65aca32efdcb (origin/master, origin/HEAD, master) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561329 8186912 622032 33370273 1fd30a1 vmlinux
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       00000000004820cb  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427d4  016527d4
       0000000000000a1c  0000000000000000   A       0     0     1

I still do not know what affects reproducibility.
(compiler version, pahole version, etc. ?)

Aurelien used GCC 12 + binutils 2.40, but
my toolchain is a bit older.

FWIW, I tested this on Ubuntu 22.04LTS.

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-gcc --version
aarch64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

masahiro@zoe:~/ref/linux(master)$ pahole --version
v1.22

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-as --version
GNU assembler (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `aarch64-linux-gnu'.

Quoted 73 lines

Strange.

I used the .config file Aurelien provided, but
I still cannot reproduce this issue.

The vmlinux size is small
as-is in the current mainline.

[mainline]

masahiro@zoe:~/ref/linux(master)$ git log --oneline -1
65aca32efdcb (HEAD -> master, origin/master, origin/HEAD) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561282 8186912 622032 33370226 1fd3072 vmlinux
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       000000000048209c  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427a4  016527a4
       0000000000000a1c  0000000000000000   A       0     0     1

[mainline + revert 994b7ac]

masahiro@zoe:~/ref/linux2(testing)$ git log --oneline -2
856c80dd789c (HEAD -> testing) Revert "arm64: remove special treatment
for the link order of head.o"
65aca32efdcb (origin/master, origin/HEAD, master) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561329 8186912 622032 33370273 1fd30a1 vmlinux
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       00000000004820cb  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427d4  016527d4
       0000000000000a1c  0000000000000000   A       0     0     1

I still do not know what affects reproducibility.
(compiler version, pahole version, etc. ?)

Aurelien used GCC 12 + binutils 2.40, but
my toolchain is a bit older.

FWIW, I tested this on Ubuntu 22.04LTS.

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-gcc --version
aarch64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

masahiro@zoe:~/ref/linux(master)$ pahole --version
v1.22

Quoted 7 lines

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-as --version
GNU assembler (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `aarch64-linux-gnu'.

I did the same things in Deiban sid
in order to use newer versions of tools.

Yup, I saw a huge increase in the .BTF section,
and observed the difference w/wo 994b7ac.

masahiro@3e9802d667e3:~/ref/linux2$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091d26c4  011e26c4
       000000000093e626  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff800009b10cec  01b20cec
       0000000000000a1c  0000000000000000   A       0     0     1

I guess some tool might be affecting this.
Even with 994b7ac reverted, the .BTF section
is much bigger.

At the same time, I saw a ton of warnings
while building BTF.

masahiro@3e9802d667e3:~/ref/linux2$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux bookworm/sid"
NAME="Debian GNU/Linux"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

LD vmlinux
BTFIDS vmlinux
WARN: multiple IDs found for 'task_struct': 177, 16690 - using 177
WARN: multiple IDs found for 'file': 517, 16712 - using 517
WARN: multiple IDs found for 'vm_area_struct': 524, 16714 - using 524
WARN: multiple IDs found for 'inode': 586, 16773 - using 586
WARN: multiple IDs found for 'path': 618, 16802 - using 618
WARN: multiple IDs found for 'task_struct': 177, 17267 - using 177
WARN: multiple IDs found for 'file': 517, 17312 - using 517
WARN: multiple IDs found for 'vm_area_struct': 524, 17315 - using 524
WARN: multiple IDs found for 'seq_file': 1029, 17376 - using 1029
WARN: multiple IDs found for 'inode': 586, 17494 - using 586
WARN: multiple IDs found for 'path': 618, 17523 - using 618
WARN: multiple IDs found for 'cgroup': 704, 17532 - using 704
WARN: multiple IDs found for 'task_struct': 177, 18652 - using 177
WARN: multiple IDs found for 'file': 517, 18704 - using 517
WARN: multiple IDs found for 'vm_area_struct': 524, 18707 - using 524
WARN: multiple IDs found for 'seq_file': 1029, 18781 - using 1029
WARN: multiple IDs found for 'inode': 586, 18911 - using 586
WARN: multiple IDs found for 'path': 618, 18940 - using 618
WARN: multiple IDs found for 'cgroup': 704, 18949 - using 704
WARN: multiple IDs found for 'task_struct': 177, 20514 - using 177
WARN: multiple IDs found for 'file': 517, 20515 - using 517
WARN: multiple IDs found for 'vm_area_struct': 524, 20541 - using 524
WARN: multiple IDs found for 'inode': 586, 20595 - using 586
WARN: multiple IDs found for 'path': 618, 20624 - using 618
WARN: multiple IDs found for 'cgroup': 704, 20639 - using 704
WARN: multiple IDs found for 'seq_file': 1029, 20801 - using 1029
...

I am not sure whether these warnings are related to the current issue or not.

I did not look into it any further.
I may not be seeing a sane build result.

[...]

Quoted 71 lines

Quoted 63 lines

Strange.

I used the .config file Aurelien provided, but
I still cannot reproduce this issue.

The vmlinux size is small as-is in the current mainline.

[mainline]

masahiro@zoe:~/ref/linux(master)$ git log --oneline -1
65aca32efdcb (HEAD -> master, origin/master, origin/HEAD) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561282 8186912 622032 33370226 1fd3072 vmlinux
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       000000000048209c  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427a4  016527a4
       0000000000000a1c  0000000000000000   A       0     0     1

[mainline + revert 994b7ac]

masahiro@zoe:~/ref/linux2(testing)$ git log --oneline -2
856c80dd789c (HEAD -> testing) Revert "arm64: remove special treatment
for the link order of head.o"
65aca32efdcb (origin/master, origin/HEAD, master) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561329 8186912 622032 33370273 1fd30a1 vmlinux
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       00000000004820cb  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427d4  016527d4
       0000000000000a1c  0000000000000000   A       0     0     1

I still do not know what affects reproducibility.
(compiler version, pahole version, etc. ?)

Aurelien used GCC 12 + binutils 2.40, but
my toolchain is a bit older.

FWIW, I tested this on Ubuntu 22.04LTS.

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-gcc --version
aarch64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

masahiro@zoe:~/ref/linux(master)$ pahole --version
v1.22

Quoted 7 lines

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-as --version
GNU assembler (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `aarch64-linux-gnu'.

Quoted 2 lines

I did the same things in Deiban sid
in order to use newer versions of tools.

Hi Masahiro,

An upgrade from gcc 11 to gcc 12, BTF section increase and a number of duplicate IDs reported by resolve_btfids matches the description of the following thread:

https://lore.kernel.org/bpf/Y%[email protected]/

The issue is caused by change in GNU assembler DWARF generation. I've sent a patch to fix it a few weeks ago and it is merged in dwarves master:

a9498899109d ("dwarf_loader: Fix for BTF id drift caused by adding unspecified types")

Could you please grab a fresh version of dwarves from:

[email protected]:acmel/dwarves.git

compile 'pahole' and try with?

Em Tue, Mar 28, 2023 at 01:33:29PM +0300, Eduard Zingerman escreveu:

Quoted 93 lines · Masahiro Yamada

[...]

Quoted 71 lines

Quoted 63 lines

Strange.

I used the .config file Aurelien provided, but
I still cannot reproduce this issue.

The vmlinux size is small as-is in the current mainline.

[mainline]

masahiro@zoe:~/ref/linux(master)$ git log --oneline -1
65aca32efdcb (HEAD -> master, origin/master, origin/HEAD) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561282 8186912 622032 33370226 1fd3072 vmlinux
masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       000000000048209c  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427a4  016527a4
       0000000000000a1c  0000000000000000   A       0     0     1

[mainline + revert 994b7ac]

masahiro@zoe:~/ref/linux2(testing)$ git log --oneline -2
856c80dd789c (HEAD -> testing) Revert "arm64: remove special treatment
for the link order of head.o"
65aca32efdcb (origin/master, origin/HEAD, master) Merge tag
'mm-hotfixes-stable-2023-03-24-17-09' of
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-size  vmlinux
   text    data     bss     dec     hex filename
24561329 8186912 622032 33370273 1fd30a1 vmlinux
masahiro@zoe:~/ref/linux2(testing)$ aarch64-linux-gnu-readelf -S
vmlinux | grep -A1 BTF
  [15] .BTF              PROGBITS         ffff8000091c0708  011d0708
       00000000004820cb  0000000000000000   A       0     0     1
  [16] .BTF_ids          PROGBITS         ffff8000096427d4  016527d4
       0000000000000a1c  0000000000000000   A       0     0     1

I still do not know what affects reproducibility.
(compiler version, pahole version, etc. ?)

Aurelien used GCC 12 + binutils 2.40, but
my toolchain is a bit older.

FWIW, I tested this on Ubuntu 22.04LTS.

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-gcc --version
aarch64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

masahiro@zoe:~/ref/linux(master)$ pahole --version
v1.22

Quoted 7 lines

masahiro@zoe:~/ref/linux(master)$ aarch64-linux-gnu-as --version
GNU assembler (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `aarch64-linux-gnu'.

Quoted 2 lines

I did the same things in Deiban sid
in order to use newer versions of tools.

Hi Masahiro,

An upgrade from gcc 11 to gcc 12, BTF section increase and a number of duplicate IDs reported by resolve_btfids matches the description of the following thread:

https://lore.kernel.org/bpf/Y%[email protected]/

The issue is caused by change in GNU assembler DWARF generation. I've sent a patch to fix it a few weeks ago and it is merged in dwarves master:

a9498899109d ("dwarf_loader: Fix for BTF id drift caused by adding unspecified types")

Could you please grab a fresh version of dwarves from:

Quoted 3 lines

[email protected]:acmel/dwarves.git

compile 'pahole' and try with?

pahole 1.25 is long overdue, so let see if this got fixed with what is in master, please take a look, you can as well get it from:

git://git.kernel.org/pub/scm/devel/pahole/pahole.git