Skip to contentSkip to conversation

btf_encoder: Handle DW_TAG_variable that has DW_AT_specification

PATCH Hao Luo 3 participants 8 messages Sep 30, 2020 – Oct 1, 2020 4 messages not in the archive BPF

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): CA+khW7jf7Z=sMC1u5eyn6XOZDTFJaNjV-D0ogvQSyUGSKjC3LQ@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

Hao Luo

author
#4 lore raw

Arnaldo,

Is this patch ready to be merged into Pahole's master branch? Alexei is testing the kernel patches that need this patch. Please let me know if there is anything I can do to help merging.

Thank you,
Hao

Quoted 167 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Hao Luo

Arnaldo,

Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Aug 24, 2020 at 05:45:23PM -0700, Hao Luo escreveu:

Quoted 7 lines

It is found on gcc 8.2 that global percpu variables generate the following dwarf entry in the cu where the variable is defined[1].

Take the global variable "bpf_prog_active" defined in kernel/bpf/syscall.c as an example. The debug info for syscall.c has two dwarf entries for "bpf_prog_active".

[...]

Quoted 4 lines

Interesting, here I get, with binutils' readelf:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 1 line
    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 5 lines

[root@quaco perf]#

Just one, as:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active -B1 -A8

Quoted 2 lines

<1><f6a0>: Abbrev Number: 103 (DW_TAG_variable)

    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 8 lines
    <f6a5>   DW_AT_decl_file   : 11
    <f6a6>   DW_AT_decl_line   : 1008
    <f6a8>   DW_AT_decl_column : 1
    <f6a9>   DW_AT_type        : <0xcf>
    <f6ad>   DW_AT_external    : 1
    <f6ad>   DW_AT_declaration : 1
 <1><f6ad>: Abbrev Number: 103 (DW_TAG_variable)
    <f6ae>   DW_AT_name        : (indirect string, offset: 0x3a5d):

bpf_stats_enabled_mutex

Quoted 5 lines

[root@quaco perf]#

I get what you have when I use elfutils' readelf:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 5 lines

name (strp) "bpf_prog_active"
[ 0] addr .data..percpu+0 <bpf_prog_active>
[root@quaco perf]#

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B1 -A8 \"bpf_prog_active\"

Quoted 10 lines
 [  f6a0]    variable             abbrev: 103
             name                 (strp) "bpf_prog_active"
             decl_file            (data1) bpf.h (11)
             decl_line            (data2) 1008
             decl_column          (data1) 1
             type                 (ref4) [    cf]
             external             (flag_present) yes
             declaration          (flag_present) yes
 [  f6ad]    variable             abbrev: 103
             name                 (strp) "bpf_stats_enabled_mutex"
Quoted 5 lines

[root@quaco perf]#

And:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B5 \<bpf_prog_active\>

Quoted 8 lines
 [ 1bdf5]    variable             abbrev: 212
             specification        (ref4) [  f6a0]
             decl_file            (data1) syscall.c (1)
             decl_line            (data1) 43
             location             (exprloc)
              [ 0] addr .data..percpu+0 <bpf_prog_active>

[root@quaco perf]#

Quoted 1 line

In binutils readelf, there is a extra entry

Not here, tomorrow I'll triple check.

Quoted 34 lines

<1><1b24c>: Abbrev Number: 195 (DW_TAG_variable)

   <1b24e>   DW_AT_specification: <0xf335>
   <1b252>   DW_AT_decl_file   : 1
   <1b253>   DW_AT_decl_line   : 43
   <1b254>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0

(DW_OP_addr: 0)

which points to

<1><f335>: Abbrev Number: 95 (DW_TAG_variable)
<f336> DW_AT_name : (indirect string, offset: 0xb37a):
bpf_prog_active

It just doesn't have the string 'bpf_prog_active', annotating entry. So eu-readelf and binutils readelf have the same results.

Quoted 5 lines
Quoted 2 lines

Note that second DW_TAG_variable entry contains specification that points to the first entry.

So you are not considering the first when encoding since it is just a DW_AT_declaration, considers the second, as it should be, and then

needs

Quoted 4 lines

to go see its DW_AT_specification, right?

Sounds correct, applying, will test further and then push out,

Yes, exactly. The var tags to be considered are those that either have DW_AT_specification or not have DW_AT_declaration. This makes sure btf_encoder works correctly on both old and new gcc.

Quoted 2 lines

Thanks,

Quoted 8 lines
Quoted 1 line

- Arnaldo

Suggested by Yonghong, I tested this change on a larger set of compilers this time and works correctly. See below.

Could you also add 'Reported-by: Yonghong Song <[email protected]>'? I should have done that when sending out this patch. The credit goes to Yonghong.

Sure, and I'll add your results with different computers, for the record.

Thanks,

- Arnaldo

Quoted 24 lines

Thank you,
Hao

clang 10:
[67] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20168] VAR 'bpf_prog_active' type_id=67, linkage=global-alloc

clang 9:
[64] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [19789] VAR 'bpf_prog_active' type_id=64, linkage=global-alloc

gcc 10.2
[18] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20319] VAR 'bpf_prog_active' type_id=18, linkage=global-alloc

gcc 9.3:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21085] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 8
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21084] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

Quoted 7 lines

gcc 6.2
[22] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21083] VAR 'bpf_prog_active' type_id=22, linkage=global-alloc

gcc 4.9
[17] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20410] VAR 'bpf_prog_active' type_id=17, linkage=global-alloc

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

#5 lore raw

Arnaldo,

ping.
Is anything blocking this fix from merging? The kernel patches are stalled waiting on the pahole.

Thanks

Quoted 179 lines · Hao Luo

Arnaldo,

Is this patch ready to be merged into Pahole's master branch? Alexei is testing the kernel patches that need this patch. Please let me know if there is anything I can do to help merging.

Thank you,
Hao

Quoted 167 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Hao Luo

Arnaldo,

Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Aug 24, 2020 at 05:45:23PM -0700, Hao Luo escreveu:

Quoted 7 lines

It is found on gcc 8.2 that global percpu variables generate the following dwarf entry in the cu where the variable is defined[1].

Take the global variable "bpf_prog_active" defined in kernel/bpf/syscall.c as an example. The debug info for syscall.c has two dwarf entries for "bpf_prog_active".

[...]

Quoted 4 lines

Interesting, here I get, with binutils' readelf:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 1 line
    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 5 lines

[root@quaco perf]#

Just one, as:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active -B1 -A8

Quoted 2 lines

<1><f6a0>: Abbrev Number: 103 (DW_TAG_variable)

    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 8 lines
    <f6a5>   DW_AT_decl_file   : 11
    <f6a6>   DW_AT_decl_line   : 1008
    <f6a8>   DW_AT_decl_column : 1
    <f6a9>   DW_AT_type        : <0xcf>
    <f6ad>   DW_AT_external    : 1
    <f6ad>   DW_AT_declaration : 1
 <1><f6ad>: Abbrev Number: 103 (DW_TAG_variable)
    <f6ae>   DW_AT_name        : (indirect string, offset: 0x3a5d):

bpf_stats_enabled_mutex

Quoted 5 lines

[root@quaco perf]#

I get what you have when I use elfutils' readelf:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 5 lines

name (strp) "bpf_prog_active"
[ 0] addr .data..percpu+0 <bpf_prog_active>
[root@quaco perf]#

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B1 -A8 \"bpf_prog_active\"

Quoted 10 lines
 [  f6a0]    variable             abbrev: 103
             name                 (strp) "bpf_prog_active"
             decl_file            (data1) bpf.h (11)
             decl_line            (data2) 1008
             decl_column          (data1) 1
             type                 (ref4) [    cf]
             external             (flag_present) yes
             declaration          (flag_present) yes
 [  f6ad]    variable             abbrev: 103
             name                 (strp) "bpf_stats_enabled_mutex"
Quoted 5 lines

[root@quaco perf]#

And:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B5 \<bpf_prog_active\>

Quoted 8 lines
 [ 1bdf5]    variable             abbrev: 212
             specification        (ref4) [  f6a0]
             decl_file            (data1) syscall.c (1)
             decl_line            (data1) 43
             location             (exprloc)
              [ 0] addr .data..percpu+0 <bpf_prog_active>

[root@quaco perf]#

Quoted 1 line

In binutils readelf, there is a extra entry

Not here, tomorrow I'll triple check.

Quoted 34 lines

<1><1b24c>: Abbrev Number: 195 (DW_TAG_variable)

   <1b24e>   DW_AT_specification: <0xf335>
   <1b252>   DW_AT_decl_file   : 1
   <1b253>   DW_AT_decl_line   : 43
   <1b254>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0

(DW_OP_addr: 0)

which points to

<1><f335>: Abbrev Number: 95 (DW_TAG_variable)
<f336> DW_AT_name : (indirect string, offset: 0xb37a):
bpf_prog_active

It just doesn't have the string 'bpf_prog_active', annotating entry. So eu-readelf and binutils readelf have the same results.

Quoted 5 lines
Quoted 2 lines

Note that second DW_TAG_variable entry contains specification that points to the first entry.

So you are not considering the first when encoding since it is just a DW_AT_declaration, considers the second, as it should be, and then

needs

Quoted 4 lines

to go see its DW_AT_specification, right?

Sounds correct, applying, will test further and then push out,

Yes, exactly. The var tags to be considered are those that either have DW_AT_specification or not have DW_AT_declaration. This makes sure btf_encoder works correctly on both old and new gcc.

Quoted 2 lines

Thanks,

Quoted 8 lines
Quoted 1 line

- Arnaldo

Suggested by Yonghong, I tested this change on a larger set of compilers this time and works correctly. See below.

Could you also add 'Reported-by: Yonghong Song <[email protected]>'? I should have done that when sending out this patch. The credit goes to Yonghong.

Sure, and I'll add your results with different computers, for the record.

Thanks,

- Arnaldo

Quoted 24 lines

Thank you,
Hao

clang 10:
[67] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20168] VAR 'bpf_prog_active' type_id=67, linkage=global-alloc

clang 9:
[64] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [19789] VAR 'bpf_prog_active' type_id=64, linkage=global-alloc

gcc 10.2
[18] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20319] VAR 'bpf_prog_active' type_id=18, linkage=global-alloc

gcc 9.3:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21085] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 8
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21084] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

Quoted 7 lines

gcc 6.2
[22] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21083] VAR 'bpf_prog_active' type_id=22, linkage=global-alloc

gcc 4.9
[17] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20410] VAR 'bpf_prog_active' type_id=17, linkage=global-alloc

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

#6 lore raw

Em Thu, Oct 01, 2020 at 08:47:51AM -0700, Alexei Starovoitov escreveu:

Quoted 5 lines

Arnaldo,

ping.
Is anything blocking this fix from merging? The kernel patches are stalled waiting on the pahole.

Applied locally, testing now, will push to the main branch ASAP.

- Arnaldo

Quoted 181 lines

Thanks

Quoted 179 lines · Hao Luo

Arnaldo,

Is this patch ready to be merged into Pahole's master branch? Alexei is testing the kernel patches that need this patch. Please let me know if there is anything I can do to help merging.

Thank you,
Hao

Quoted 167 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Hao Luo

Arnaldo,

Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Aug 24, 2020 at 05:45:23PM -0700, Hao Luo escreveu:

Quoted 7 lines

It is found on gcc 8.2 that global percpu variables generate the following dwarf entry in the cu where the variable is defined[1].

Take the global variable "bpf_prog_active" defined in kernel/bpf/syscall.c as an example. The debug info for syscall.c has two dwarf entries for "bpf_prog_active".

[...]

Quoted 4 lines

Interesting, here I get, with binutils' readelf:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 1 line
    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 5 lines

[root@quaco perf]#

Just one, as:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active -B1 -A8

Quoted 2 lines

<1><f6a0>: Abbrev Number: 103 (DW_TAG_variable)

    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 8 lines
    <f6a5>   DW_AT_decl_file   : 11
    <f6a6>   DW_AT_decl_line   : 1008
    <f6a8>   DW_AT_decl_column : 1
    <f6a9>   DW_AT_type        : <0xcf>
    <f6ad>   DW_AT_external    : 1
    <f6ad>   DW_AT_declaration : 1
 <1><f6ad>: Abbrev Number: 103 (DW_TAG_variable)
    <f6ae>   DW_AT_name        : (indirect string, offset: 0x3a5d):

bpf_stats_enabled_mutex

Quoted 5 lines

[root@quaco perf]#

I get what you have when I use elfutils' readelf:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 5 lines

name (strp) "bpf_prog_active"
[ 0] addr .data..percpu+0 <bpf_prog_active>
[root@quaco perf]#

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B1 -A8 \"bpf_prog_active\"

Quoted 10 lines
 [  f6a0]    variable             abbrev: 103
             name                 (strp) "bpf_prog_active"
             decl_file            (data1) bpf.h (11)
             decl_line            (data2) 1008
             decl_column          (data1) 1
             type                 (ref4) [    cf]
             external             (flag_present) yes
             declaration          (flag_present) yes
 [  f6ad]    variable             abbrev: 103
             name                 (strp) "bpf_stats_enabled_mutex"
Quoted 5 lines

[root@quaco perf]#

And:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B5 \<bpf_prog_active\>

Quoted 8 lines
 [ 1bdf5]    variable             abbrev: 212
             specification        (ref4) [  f6a0]
             decl_file            (data1) syscall.c (1)
             decl_line            (data1) 43
             location             (exprloc)
              [ 0] addr .data..percpu+0 <bpf_prog_active>

[root@quaco perf]#

Quoted 1 line

In binutils readelf, there is a extra entry

Not here, tomorrow I'll triple check.

Quoted 34 lines

<1><1b24c>: Abbrev Number: 195 (DW_TAG_variable)

   <1b24e>   DW_AT_specification: <0xf335>
   <1b252>   DW_AT_decl_file   : 1
   <1b253>   DW_AT_decl_line   : 43
   <1b254>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0

(DW_OP_addr: 0)

which points to

<1><f335>: Abbrev Number: 95 (DW_TAG_variable)
<f336> DW_AT_name : (indirect string, offset: 0xb37a):
bpf_prog_active

It just doesn't have the string 'bpf_prog_active', annotating entry. So eu-readelf and binutils readelf have the same results.

Quoted 5 lines
Quoted 2 lines

Note that second DW_TAG_variable entry contains specification that points to the first entry.

So you are not considering the first when encoding since it is just a DW_AT_declaration, considers the second, as it should be, and then

needs

Quoted 4 lines

to go see its DW_AT_specification, right?

Sounds correct, applying, will test further and then push out,

Yes, exactly. The var tags to be considered are those that either have DW_AT_specification or not have DW_AT_declaration. This makes sure btf_encoder works correctly on both old and new gcc.

Quoted 2 lines

Thanks,

Quoted 8 lines
Quoted 1 line

- Arnaldo

Suggested by Yonghong, I tested this change on a larger set of compilers this time and works correctly. See below.

Could you also add 'Reported-by: Yonghong Song <[email protected]>'? I should have done that when sending out this patch. The credit goes to Yonghong.

Sure, and I'll add your results with different computers, for the record.

Thanks,

- Arnaldo

Quoted 24 lines

Thank you,
Hao

clang 10:
[67] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20168] VAR 'bpf_prog_active' type_id=67, linkage=global-alloc

clang 9:
[64] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [19789] VAR 'bpf_prog_active' type_id=64, linkage=global-alloc

gcc 10.2
[18] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20319] VAR 'bpf_prog_active' type_id=18, linkage=global-alloc

gcc 9.3:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21085] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 8
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21084] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

Quoted 7 lines

gcc 6.2
[22] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21083] VAR 'bpf_prog_active' type_id=22, linkage=global-alloc

gcc 4.9
[17] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20410] VAR 'bpf_prog_active' type_id=17, linkage=global-alloc

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Signature · 1 line
- Arnaldo
author
#7 lore raw

Thanks!

Quoted 197 lines · Arnaldo Carvalho de Melo

Em Thu, Oct 01, 2020 at 08:47:51AM -0700, Alexei Starovoitov escreveu:

Quoted 5 lines

Arnaldo,

ping.
Is anything blocking this fix from merging? The kernel patches are stalled waiting on the pahole.

Applied locally, testing now, will push to the main branch ASAP.

- Arnaldo

Quoted 181 lines

Thanks

Quoted 179 lines · Hao Luo

Arnaldo,

Is this patch ready to be merged into Pahole's master branch? Alexei is testing the kernel patches that need this patch. Please let me know if there is anything I can do to help merging.

Thank you,
Hao

Quoted 167 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Hao Luo

Arnaldo,

Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Aug 24, 2020 at 05:45:23PM -0700, Hao Luo escreveu:

Quoted 7 lines

It is found on gcc 8.2 that global percpu variables generate the following dwarf entry in the cu where the variable is defined[1].

Take the global variable "bpf_prog_active" defined in kernel/bpf/syscall.c as an example. The debug info for syscall.c has two dwarf entries for "bpf_prog_active".

[...]

Quoted 4 lines

Interesting, here I get, with binutils' readelf:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 1 line
    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 5 lines

[root@quaco perf]#

Just one, as:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active -B1 -A8

Quoted 2 lines

<1><f6a0>: Abbrev Number: 103 (DW_TAG_variable)

    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 8 lines
    <f6a5>   DW_AT_decl_file   : 11
    <f6a6>   DW_AT_decl_line   : 1008
    <f6a8>   DW_AT_decl_column : 1
    <f6a9>   DW_AT_type        : <0xcf>
    <f6ad>   DW_AT_external    : 1
    <f6ad>   DW_AT_declaration : 1
 <1><f6ad>: Abbrev Number: 103 (DW_TAG_variable)
    <f6ae>   DW_AT_name        : (indirect string, offset: 0x3a5d):

bpf_stats_enabled_mutex

Quoted 5 lines

[root@quaco perf]#

I get what you have when I use elfutils' readelf:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 5 lines

name (strp) "bpf_prog_active"
[ 0] addr .data..percpu+0 <bpf_prog_active>
[root@quaco perf]#

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B1 -A8 \"bpf_prog_active\"

Quoted 10 lines
 [  f6a0]    variable             abbrev: 103
             name                 (strp) "bpf_prog_active"
             decl_file            (data1) bpf.h (11)
             decl_line            (data2) 1008
             decl_column          (data1) 1
             type                 (ref4) [    cf]
             external             (flag_present) yes
             declaration          (flag_present) yes
 [  f6ad]    variable             abbrev: 103
             name                 (strp) "bpf_stats_enabled_mutex"
Quoted 5 lines

[root@quaco perf]#

And:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B5 \<bpf_prog_active\>

Quoted 8 lines
 [ 1bdf5]    variable             abbrev: 212
             specification        (ref4) [  f6a0]
             decl_file            (data1) syscall.c (1)
             decl_line            (data1) 43
             location             (exprloc)
              [ 0] addr .data..percpu+0 <bpf_prog_active>

[root@quaco perf]#

Quoted 1 line

In binutils readelf, there is a extra entry

Not here, tomorrow I'll triple check.

Quoted 34 lines

<1><1b24c>: Abbrev Number: 195 (DW_TAG_variable)

   <1b24e>   DW_AT_specification: <0xf335>
   <1b252>   DW_AT_decl_file   : 1
   <1b253>   DW_AT_decl_line   : 43
   <1b254>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0

(DW_OP_addr: 0)

which points to

<1><f335>: Abbrev Number: 95 (DW_TAG_variable)
<f336> DW_AT_name : (indirect string, offset: 0xb37a):
bpf_prog_active

It just doesn't have the string 'bpf_prog_active', annotating entry. So eu-readelf and binutils readelf have the same results.

Quoted 5 lines
Quoted 2 lines

Note that second DW_TAG_variable entry contains specification that points to the first entry.

So you are not considering the first when encoding since it is just a DW_AT_declaration, considers the second, as it should be, and then

needs

Quoted 4 lines

to go see its DW_AT_specification, right?

Sounds correct, applying, will test further and then push out,

Yes, exactly. The var tags to be considered are those that either have DW_AT_specification or not have DW_AT_declaration. This makes sure btf_encoder works correctly on both old and new gcc.

Quoted 2 lines

Thanks,

Quoted 8 lines
Quoted 1 line

- Arnaldo

Suggested by Yonghong, I tested this change on a larger set of compilers this time and works correctly. See below.

Could you also add 'Reported-by: Yonghong Song <[email protected]>'? I should have done that when sending out this patch. The credit goes to Yonghong.

Sure, and I'll add your results with different computers, for the record.

Thanks,

- Arnaldo

Quoted 24 lines

Thank you,
Hao

clang 10:
[67] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20168] VAR 'bpf_prog_active' type_id=67, linkage=global-alloc

clang 9:
[64] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [19789] VAR 'bpf_prog_active' type_id=64, linkage=global-alloc

gcc 10.2
[18] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20319] VAR 'bpf_prog_active' type_id=18, linkage=global-alloc

gcc 9.3:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21085] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 8
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21084] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

Quoted 7 lines

gcc 6.2
[22] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21083] VAR 'bpf_prog_active' type_id=22, linkage=global-alloc

gcc 4.9
[17] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20410] VAR 'bpf_prog_active' type_id=17, linkage=global-alloc

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--

- Arnaldo

#8 lore raw

Em Thu, Oct 01, 2020 at 11:40:25AM -0700, Hao Luo escreveu:

Quoted 1 line

Thanks!

I must just apologise not having this in an officially released version yet, getting constantly postponed due to bug reports about corner cases and some features I got carried away working on, I'll fast pace a new version to avoid getting in the way of the larger eBPF effort.

- Arnaldo

Quoted 199 lines
Quoted 197 lines · Arnaldo Carvalho de Melo

Em Thu, Oct 01, 2020 at 08:47:51AM -0700, Alexei Starovoitov escreveu:

Quoted 5 lines

Arnaldo,

ping.
Is anything blocking this fix from merging? The kernel patches are stalled waiting on the pahole.

Applied locally, testing now, will push to the main branch ASAP.

- Arnaldo

Quoted 181 lines

Thanks

Quoted 179 lines · Hao Luo

Arnaldo,

Is this patch ready to be merged into Pahole's master branch? Alexei is testing the kernel patches that need this patch. Please let me know if there is anything I can do to help merging.

Thank you,
Hao

Quoted 167 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Hao Luo

Arnaldo,

Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Aug 24, 2020 at 05:45:23PM -0700, Hao Luo escreveu:

Quoted 7 lines

It is found on gcc 8.2 that global percpu variables generate the following dwarf entry in the cu where the variable is defined[1].

Take the global variable "bpf_prog_active" defined in kernel/bpf/syscall.c as an example. The debug info for syscall.c has two dwarf entries for "bpf_prog_active".

[...]

Quoted 4 lines

Interesting, here I get, with binutils' readelf:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 1 line
    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 5 lines

[root@quaco perf]#

Just one, as:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active -B1 -A8

Quoted 2 lines

<1><f6a0>: Abbrev Number: 103 (DW_TAG_variable)

    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 8 lines
    <f6a5>   DW_AT_decl_file   : 11
    <f6a6>   DW_AT_decl_line   : 1008
    <f6a8>   DW_AT_decl_column : 1
    <f6a9>   DW_AT_type        : <0xcf>
    <f6ad>   DW_AT_external    : 1
    <f6ad>   DW_AT_declaration : 1
 <1><f6ad>: Abbrev Number: 103 (DW_TAG_variable)
    <f6ae>   DW_AT_name        : (indirect string, offset: 0x3a5d):

bpf_stats_enabled_mutex

Quoted 5 lines

[root@quaco perf]#

I get what you have when I use elfutils' readelf:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 5 lines

name (strp) "bpf_prog_active"
[ 0] addr .data..percpu+0 <bpf_prog_active>
[root@quaco perf]#

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B1 -A8 \"bpf_prog_active\"

Quoted 10 lines
 [  f6a0]    variable             abbrev: 103
             name                 (strp) "bpf_prog_active"
             decl_file            (data1) bpf.h (11)
             decl_line            (data2) 1008
             decl_column          (data1) 1
             type                 (ref4) [    cf]
             external             (flag_present) yes
             declaration          (flag_present) yes
 [  f6ad]    variable             abbrev: 103
             name                 (strp) "bpf_stats_enabled_mutex"
Quoted 5 lines

[root@quaco perf]#

And:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B5 \<bpf_prog_active\>

Quoted 8 lines
 [ 1bdf5]    variable             abbrev: 212
             specification        (ref4) [  f6a0]
             decl_file            (data1) syscall.c (1)
             decl_line            (data1) 43
             location             (exprloc)
              [ 0] addr .data..percpu+0 <bpf_prog_active>

[root@quaco perf]#

Quoted 1 line

In binutils readelf, there is a extra entry

Not here, tomorrow I'll triple check.

Quoted 34 lines

<1><1b24c>: Abbrev Number: 195 (DW_TAG_variable)

   <1b24e>   DW_AT_specification: <0xf335>
   <1b252>   DW_AT_decl_file   : 1
   <1b253>   DW_AT_decl_line   : 43
   <1b254>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0

(DW_OP_addr: 0)

which points to

<1><f335>: Abbrev Number: 95 (DW_TAG_variable)
<f336> DW_AT_name : (indirect string, offset: 0xb37a):
bpf_prog_active

It just doesn't have the string 'bpf_prog_active', annotating entry. So eu-readelf and binutils readelf have the same results.

Quoted 5 lines
Quoted 2 lines

Note that second DW_TAG_variable entry contains specification that points to the first entry.

So you are not considering the first when encoding since it is just a DW_AT_declaration, considers the second, as it should be, and then

needs

Quoted 4 lines

to go see its DW_AT_specification, right?

Sounds correct, applying, will test further and then push out,

Yes, exactly. The var tags to be considered are those that either have DW_AT_specification or not have DW_AT_declaration. This makes sure btf_encoder works correctly on both old and new gcc.

Quoted 2 lines

Thanks,

Quoted 8 lines
Quoted 1 line

- Arnaldo

Suggested by Yonghong, I tested this change on a larger set of compilers this time and works correctly. See below.

Could you also add 'Reported-by: Yonghong Song <[email protected]>'? I should have done that when sending out this patch. The credit goes to Yonghong.

Sure, and I'll add your results with different computers, for the record.

Thanks,

- Arnaldo

Quoted 24 lines

Thank you,
Hao

clang 10:
[67] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20168] VAR 'bpf_prog_active' type_id=67, linkage=global-alloc

clang 9:
[64] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [19789] VAR 'bpf_prog_active' type_id=64, linkage=global-alloc

gcc 10.2
[18] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20319] VAR 'bpf_prog_active' type_id=18, linkage=global-alloc

gcc 9.3:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21085] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 8
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21084] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

Quoted 7 lines

gcc 6.2
[22] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21083] VAR 'bpf_prog_active' type_id=22, linkage=global-alloc

gcc 4.9
[17] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20410] VAR 'bpf_prog_active' type_id=17, linkage=global-alloc

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--

- Arnaldo

Signature · 1 line
- Arnaldo
author
#9 lore raw

Arnaldo, thanks for the update. In that case, I think on the kernel side I need to skip encoding percpu vars for this pahole release, and re-enable for the next pahole release. (assuming the flag for opt-out is in this release). Alexei, do you have any better idea?

Hao

Quoted 214 lines · Arnaldo Carvalho de Melo

Em Thu, Oct 01, 2020 at 11:40:25AM -0700, Hao Luo escreveu:

Quoted 1 line

Thanks!

I must just apologise not having this in an officially released version yet, getting constantly postponed due to bug reports about corner cases and some features I got carried away working on, I'll fast pace a new version to avoid getting in the way of the larger eBPF effort.

- Arnaldo

Quoted 193 lines
Quoted 191 lines · Arnaldo Carvalho de Melo

Em Thu, Oct 01, 2020 at 08:47:51AM -0700, Alexei Starovoitov escreveu:

Quoted 5 lines

Arnaldo,

ping.
Is anything blocking this fix from merging? The kernel patches are stalled waiting on the pahole.

Applied locally, testing now, will push to the main branch ASAP.

- Arnaldo

Quoted 179 lines

Thanks

Quoted 177 lines · Hao Luo

Arnaldo,

Is this patch ready to be merged into Pahole's master branch? Alexei is testing the kernel patches that need this patch. Please let me know if there is anything I can do to help merging.

Thank you,
Hao

Quoted 165 lines · Arnaldo Carvalho de Melo
Quoted 77 lines · Hao Luo

Arnaldo,

Quoted 9 lines · Arnaldo Carvalho de Melo

Em Mon, Aug 24, 2020 at 05:45:23PM -0700, Hao Luo escreveu:

Quoted 7 lines

It is found on gcc 8.2 that global percpu variables generate the following dwarf entry in the cu where the variable is defined[1].

Take the global variable "bpf_prog_active" defined in kernel/bpf/syscall.c as an example. The debug info for syscall.c has two dwarf entries for "bpf_prog_active".

[...]

Quoted 4 lines

Interesting, here I get, with binutils' readelf:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 1 line
    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 5 lines

[root@quaco perf]#

Just one, as:

[root@quaco perf]# readelf -wi

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active -B1 -A8

Quoted 2 lines

<1><f6a0>: Abbrev Number: 103 (DW_TAG_variable)

    <f6a1>   DW_AT_name        : (indirect string, offset: 0xb70d):

bpf_prog_active

Quoted 8 lines
    <f6a5>   DW_AT_decl_file   : 11
    <f6a6>   DW_AT_decl_line   : 1008
    <f6a8>   DW_AT_decl_column : 1
    <f6a9>   DW_AT_type        : <0xcf>
    <f6ad>   DW_AT_external    : 1
    <f6ad>   DW_AT_declaration : 1
 <1><f6ad>: Abbrev Number: 103 (DW_TAG_variable)
    <f6ae>   DW_AT_name        : (indirect string, offset: 0x3a5d):

bpf_stats_enabled_mutex

Quoted 5 lines

[root@quaco perf]#

I get what you have when I use elfutils' readelf:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep bpf_prog_active

Quoted 5 lines

name (strp) "bpf_prog_active"
[ 0] addr .data..percpu+0 <bpf_prog_active>
[root@quaco perf]#

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B1 -A8 \"bpf_prog_active\"

Quoted 10 lines
 [  f6a0]    variable             abbrev: 103
             name                 (strp) "bpf_prog_active"
             decl_file            (data1) bpf.h (11)
             decl_line            (data2) 1008
             decl_column          (data1) 1
             type                 (ref4) [    cf]
             external             (flag_present) yes
             declaration          (flag_present) yes
 [  f6ad]    variable             abbrev: 103
             name                 (strp) "bpf_stats_enabled_mutex"
Quoted 5 lines

[root@quaco perf]#

And:

[root@quaco perf]# eu-readelf -winfo

../build/v5.8-rc5+/kernel/bpf/syscall.o | grep -B5 \<bpf_prog_active\>

Quoted 8 lines
 [ 1bdf5]    variable             abbrev: 212
             specification        (ref4) [  f6a0]
             decl_file            (data1) syscall.c (1)
             decl_line            (data1) 43
             location             (exprloc)
              [ 0] addr .data..percpu+0 <bpf_prog_active>

[root@quaco perf]#

Quoted 1 line

In binutils readelf, there is a extra entry

Not here, tomorrow I'll triple check.

Quoted 34 lines

<1><1b24c>: Abbrev Number: 195 (DW_TAG_variable)

   <1b24e>   DW_AT_specification: <0xf335>
   <1b252>   DW_AT_decl_file   : 1
   <1b253>   DW_AT_decl_line   : 43
   <1b254>   DW_AT_location    : 9 byte block: 3 0 0 0 0 0 0 0 0

(DW_OP_addr: 0)

which points to

<1><f335>: Abbrev Number: 95 (DW_TAG_variable)
<f336> DW_AT_name : (indirect string, offset: 0xb37a):
bpf_prog_active

It just doesn't have the string 'bpf_prog_active', annotating entry. So eu-readelf and binutils readelf have the same results.

Quoted 5 lines
Quoted 2 lines

Note that second DW_TAG_variable entry contains specification that points to the first entry.

So you are not considering the first when encoding since it is just a DW_AT_declaration, considers the second, as it should be, and then

needs

Quoted 4 lines

to go see its DW_AT_specification, right?

Sounds correct, applying, will test further and then push out,

Yes, exactly. The var tags to be considered are those that either have DW_AT_specification or not have DW_AT_declaration. This makes sure btf_encoder works correctly on both old and new gcc.

Quoted 2 lines

Thanks,

Quoted 8 lines
Quoted 1 line

- Arnaldo

Suggested by Yonghong, I tested this change on a larger set of compilers this time and works correctly. See below.

Could you also add 'Reported-by: Yonghong Song <[email protected]>'? I should have done that when sending out this patch. The credit goes to Yonghong.

Sure, and I'll add your results with different computers, for the record.

Thanks,

- Arnaldo

Quoted 31 lines

Thank you,
Hao

clang 10:
[67] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20168] VAR 'bpf_prog_active' type_id=67, linkage=global-alloc

clang 9:
[64] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [19789] VAR 'bpf_prog_active' type_id=64, linkage=global-alloc

gcc 10.2
[18] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20319] VAR 'bpf_prog_active' type_id=18, linkage=global-alloc

gcc 9.3:
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21085] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 8
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21084] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc

gcc 6.2
[22] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [21083] VAR 'bpf_prog_active' type_id=22, linkage=global-alloc

gcc 4.9
[17] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [20410] VAR 'bpf_prog_active' type_id=17, linkage=global-alloc

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--

- Arnaldo

--

- Arnaldo

#10 lore raw
Quoted 5 lines · Hao Luo

Arnaldo, thanks for the update. In that case, I think on the kernel side I need to skip encoding percpu vars for this pahole release, and re-enable for the next pahole release. (assuming the flag for opt-out is in this release). Alexei, do you have any better idea?

I'm not following. Let's get this fix landed in pahole and release new 1.18 with it.
The opt-out flag is orthogonal. I can be done in 1.19 or whenever. With your kernel patches the kernel will reject percpu vars when pahole is too old, because they will not be found in vmlinux btf,
so I don't see any compatibility issues.
There is no need to bump the required min version of pahole in scripts/link-vmlinux.sh.
It can stay as v1.16. We only need clean verifier message that percpu BTF is not found and the kernel needs to be rebuilt with pahole 1.18.

author
#11 lore raw
Quoted 10 lines · Alexei Starovoitov
Quoted 5 lines · Hao Luo

Arnaldo, thanks for the update. In that case, I think on the kernel side I need to skip encoding percpu vars for this pahole release, and re-enable for the next pahole release. (assuming the flag for opt-out is in this release). Alexei, do you have any better idea?

I'm not following. Let's get this fix landed in pahole and release new 1.18 with it.

Sorry, I misunderstood Arnaldo. I thought what he was saying was, not having this fix in an officially released version (I took this as the new 1.18) and will get 1.19 quicker. It would be ideal if we can have this in 1.18. Sorry for my bad reading.

Quoted 8 lines

The opt-out flag is orthogonal. I can be done in 1.19 or whenever. With your kernel patches the kernel will reject percpu vars when pahole is too old, because they will not be found in vmlinux btf,
so I don't see any compatibility issues.
There is no need to bump the required min version of pahole in scripts/link-vmlinux.sh.
It can stay as v1.16. We only need clean verifier message that percpu BTF is not found and the kernel needs to be rebuilt with pahole 1.18.