Skip to contentSkip to conversation

perf test case 40 LLVM search and compile runs endless (eBPF linkage issue)

DISCUSSION Thomas Richter 3 participants 5 messages May 23, 2022 – May 25, 2022

lore
Share & tools
lore
#0 lore raw

On s390 the test case 40 'LLVM search and compile' runs for ever:

# date; timeout 30s perf test -F 40; date
Mon May 23 12:35:48 PM CEST 2022
40: LLVM search and compile :
40.1: Basic BPF llvm compile :
Mon May 23 12:36:18 PM CEST 2022
#

Investigation by Ilya Leoshkevich revealed:
On s390 we hit an endless loop where bpf_object__next_program and bpf_program__next infinitely call each other. We tracked this down to ld.so preferring the perf's bpf_object__next_program weak copy over the libbpf's strong one.

This is introduced with commit 2a4898fc264a ("perf tools: Add more weak libbpf functions")

Also see discussion in
https://lore.kernel.org/lkml/YY%[email protected]/

How can this be fixed asap? Thanks a lot.

Signature · 5 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
--
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschäftsführung: David Faller
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
#1 lore raw
Quoted 14 lines · Thomas Richter

On s390 the test case 40 'LLVM search and compile' runs for ever:

# date; timeout 30s perf test -F 40; date
Mon May 23 12:35:48 PM CEST 2022
40: LLVM search and compile :
40.1: Basic BPF llvm compile :
Mon May 23 12:36:18 PM CEST 2022
#

Investigation by Ilya Leoshkevich revealed:
On s390 we hit an endless loop where bpf_object__next_program and bpf_program__next infinitely call each other. We tracked this down to ld.so preferring the perf's bpf_object__next_program weak copy over the libbpf's strong one.

Quoted 6 lines

This is introduced with commit 2a4898fc264a ("perf tools: Add more weak libbpf functions")

Also see discussion in
https://lore.kernel.org/lkml/YY%[email protected]/

How can this be fixed asap? Thanks a lot.

I was just suggesting to remove that:
https://lore.kernel.org/bpf/Yos8hq3NmBwemoJw@krava/T/#mda8abccbf475b99d25abe5498a5df1af2da82ee8

it's no longer needed and can cause problem apparently

jirka

#2 lore raw
Quoted 20 lines · Jiri Olsa
Quoted 19 lines · Thomas Richter

On s390 the test case 40 'LLVM search and compile' runs for ever:

# date; timeout 30s perf test -F 40; date
Mon May 23 12:35:48 PM CEST 2022
40: LLVM search and compile :
40.1: Basic BPF llvm compile :
Mon May 23 12:36:18 PM CEST 2022
#

Investigation by Ilya Leoshkevich revealed:
On s390 we hit an endless loop where bpf_object__next_program and bpf_program__next infinitely call each other. We tracked this down to ld.so preferring the perf's bpf_object__next_program weak copy over the libbpf's strong one.

This is introduced with commit 2a4898fc264a ("perf tools: Add more weak libbpf functions")

Also see discussion in
https://lore.kernel.org/lkml/YY%[email protected]/

Quoted 6 lines
Quoted 1 line

How can this be fixed asap? Thanks a lot.

I was just suggesting to remove that:
https://lore.kernel.org/bpf/Yos8hq3NmBwemoJw@krava/T/#mda8abccbf475b99d25abe5498a5df1af2da82ee8

it's no longer needed and can cause problem apparently

after discussing with Arnaldo we decided to go for another fix, could you please try this branch:

git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/weak

thanks,
jirka

#3 lore raw

Em Tue, May 24, 2022 at 02:10:17PM +0200, Jiri Olsa escreveu:

Quoted 26 lines · Jiri Olsa
Quoted 25 lines · Thomas Richter
Quoted 14 lines

On s390 the test case 40 'LLVM search and compile' runs for ever:

# date; timeout 30s perf test -F 40; date
Mon May 23 12:35:48 PM CEST 2022
40: LLVM search and compile :
40.1: Basic BPF llvm compile :
Mon May 23 12:36:18 PM CEST 2022
#

Investigation by Ilya Leoshkevich revealed:
On s390 we hit an endless loop where bpf_object__next_program and bpf_program__next infinitely call each other. We tracked this down to ld.so preferring the perf's bpf_object__next_program weak copy over the libbpf's strong one.

Quoted 6 lines

This is introduced with commit 2a4898fc264a ("perf tools: Add more weak libbpf functions")

Also see discussion in
https://lore.kernel.org/lkml/YY%[email protected]/

How can this be fixed asap? Thanks a lot.

I was just suggesting to remove that:
https://lore.kernel.org/bpf/Yos8hq3NmBwemoJw@krava/T/#mda8abccbf475b99d25abe5498a5df1af2da82ee8

Quoted 7 lines
Quoted 1 line

it's no longer needed and can cause problem apparently

after discussing with Arnaldo we decided to go for another fix, could you please try this branch:

git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/weak

I'm adding comments, testing and applying that series in my local tree, so far so good, will push to tmp.perf/core later.

Please let us know if you guys looked at the patches in Jiri's perf/weak branch and if you are ok with it so that I can add Tested-by/acked-by/reviewed-by, which are always more than welcome.

Thanks,

- Arnaldo

#4 lore raw
Quoted 38 lines · Arnaldo Carvalho de Melo

Em Tue, May 24, 2022 at 02:10:17PM +0200, Jiri Olsa escreveu:

Quoted 26 lines · Jiri Olsa
Quoted 25 lines · Thomas Richter
Quoted 14 lines

On s390 the test case 40 'LLVM search and compile' runs for ever:

# date; timeout 30s perf test -F 40; date
Mon May 23 12:35:48 PM CEST 2022
40: LLVM search and compile :
40.1: Basic BPF llvm compile :
Mon May 23 12:36:18 PM CEST 2022
#

Investigation by Ilya Leoshkevich revealed:
On s390 we hit an endless loop where bpf_object__next_program and bpf_program__next infinitely call each other. We tracked this down to ld.so preferring the perf's bpf_object__next_program weak copy over the libbpf's strong one.

Quoted 6 lines

This is introduced with commit 2a4898fc264a ("perf tools: Add more weak libbpf functions")

Also see discussion in
https://lore.kernel.org/lkml/YY%[email protected]/

How can this be fixed asap? Thanks a lot.

I was just suggesting to remove that:
https://lore.kernel.org/bpf/Yos8hq3NmBwemoJw@krava/T/#mda8abccbf475b99d25abe5498a5df1af2da82ee8

Quoted 7 lines
Quoted 1 line

it's no longer needed and can cause problem apparently

after discussing with Arnaldo we decided to go for another fix, could you please try this branch:

git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/weak

I'm adding comments, testing and applying that series in my local tree, so far so good, will push to tmp.perf/core later.

Quoted 7 lines

Please let us know if you guys looked at the patches in Jiri's perf/weak branch and if you are ok with it so that I can add Tested-by/acked-by/reviewed-by, which are always more than welcome.

Thanks,

- Arnaldo

I have taken the repo and branch perf/weak and run it through our build process. It worked fine and you have my Tested-by:

Thanks

Signature · 5 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
--
Vorsitzender des Aufsichtsrats: Gregor Pillen
Geschäftsführung: David Faller
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294