libbpf packaging

DISCUSSION Jiri Olsa 5 participants 9 messages Mar 25, 2019 – Mar 27, 2019 BPF

Share & tools

Jiri Olsa

author

2019-03-25 12:21 UTC #0 lore raw

hi guys,
we want to package libbpf and I'd like to coordinate with you on some issues I've met on this:

1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel, which would look like below (from early rpm build):

   $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
   /usr/lib/.build-id
   /usr/lib/.build-id/ca
   /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
   /usr/lib/.build-id/f6
   /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
   /usr/lib64/libbpf.so.0
   /usr/lib64/libbpf.so.0.0.1
   /usr/lib64/libcpupower.so.0
   /usr/lib64/libcpupower.so.0.0.1
   /usr/share/licenses/kernel-tools-libs
   /usr/share/licenses/kernel-tools-libs/COPYING

   $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
   /usr/include/bpf/bpf.h
   /usr/include/bpf/btf.h
   /usr/include/bpf/libbpf.h
   /usr/include/cpufreq.h
   /usr/include/cpuidle.h
   /usr/lib64/libbpf.a
   /usr/lib64/libbpf.so
   /usr/lib64/libcpupower.so

Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?

2) There's already bcc-devel's libbpf library packaged:

   $ rpm -qf /usr/lib64/libbpf.so
   bcc-devel-0.8.0-1.fc28.x86_64

so there's a conflict.. any chance we could rename libbpf to something else like:

libbpf2.so
libbpfobject.so
libbpfbest.so
...?

I checked and I think those 2 conflicting libraries don't make a valid case for using 'alternatives'.

Also the libbpf.so from bcc-devel has been there for some time so we can't just remove/rename it.. ;-)

thoughts?

thanks,
jirka

Daniel Borkmann

maintainer

2019-03-25 12:53 UTC #1 lore raw

Quoted 30 lines · Jiri Olsa

hi guys,
we want to package libbpf and I'd like to coordinate with you on some issues I've met on this:

1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel, which would look like below (from early rpm build):

   $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
   /usr/lib/.build-id
   /usr/lib/.build-id/ca
   /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
   /usr/lib/.build-id/f6
   /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
   /usr/lib64/libbpf.so.0
   /usr/lib64/libbpf.so.0.0.1
   /usr/lib64/libcpupower.so.0
   /usr/lib64/libcpupower.so.0.0.1
   /usr/share/licenses/kernel-tools-libs
   /usr/share/licenses/kernel-tools-libs/COPYING

   $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
   /usr/include/bpf/bpf.h
   /usr/include/bpf/btf.h
   /usr/include/bpf/libbpf.h
   /usr/include/cpufreq.h
   /usr/include/cpuidle.h
   /usr/lib64/libbpf.a
   /usr/lib64/libbpf.so
   /usr/lib64/libcpupower.so

Quoted 1 line

Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?

My preference is definitely on making libbpf a stand-alone package, so people can just install 'libbpf' or 'libbpf-dev{,el}' and are good to go. Also given the pace it's growing these days, it absolutely qualifies as a stand-alone package.

Quoted 5 lines

2) There's already bcc-devel's libbpf library packaged:
   $ rpm -qf /usr/lib64/libbpf.so
   bcc-devel-0.8.0-1.fc28.x86_64

Quoted 7 lines

so there's a conflict.. any chance we could rename libbpf to something else like:
   libbpf2.so
   libbpfobject.so
   libbpfbest.so
   ...?

I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.

I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.

Quoted 11 lines

I checked and I think those 2 conflicting libraries don't make a valid case for using 'alternatives'.
Also the libbpf.so from bcc-devel has been there for some time so we can't just remove/rename it.. ;-)
thoughts?
thanks,
jirka

Alexei Starovoitov

2019-03-25 16:03 UTC #2 lore raw

Quoted 32 lines · Daniel Borkmann

Quoted 30 lines · Jiri Olsa

hi guys,
we want to package libbpf and I'd like to coordinate with you on some issues I've met on this:

1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel, which would look like below (from early rpm build):

   $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
   /usr/lib/.build-id
   /usr/lib/.build-id/ca
   /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
   /usr/lib/.build-id/f6
   /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
   /usr/lib64/libbpf.so.0
   /usr/lib64/libbpf.so.0.0.1
   /usr/lib64/libcpupower.so.0
   /usr/lib64/libcpupower.so.0.0.1
   /usr/share/licenses/kernel-tools-libs
   /usr/share/licenses/kernel-tools-libs/COPYING

   $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
   /usr/include/bpf/bpf.h
   /usr/include/bpf/btf.h
   /usr/include/bpf/libbpf.h
   /usr/include/cpufreq.h
   /usr/include/cpuidle.h
   /usr/lib64/libbpf.a
   /usr/lib64/libbpf.so
   /usr/lib64/libcpupower.so

Quoted 6 lines

Quoted 1 line
Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?
My preference is definitely on making libbpf a stand-alone package, so people can just install 'libbpf' or 'libbpf-dev{,el}' and are good to go. Also given the pace it's growing these days, it absolutely qualifies as a stand-alone package.

+1
libbpf should be standalone package and not part of kernel-tools.

Quoted 13 lines

Quoted 5 lines
2) There's already bcc-devel's libbpf library packaged:
   $ rpm -qf /usr/lib64/libbpf.so
   bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
   libbpf2.so
   libbpfobject.so
   libbpfbest.so
   ...?

Quoted 7 lines

I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.

bcc migrated to libbpf some time ago.

Jiri Olsa

author

2019-03-25 17:19 UTC #3 lore raw

Quoted 35 lines · Alexei Starovoitov

Quoted 34 lines · Daniel Borkmann
Quoted 31 lines · Jiri Olsa
hi guys,
we want to package libbpf and I'd like to coordinate with you on some issues I've met on this:
1) I think libbpf should be part of kernel-tools-libs and kernel-tools-libs-devel, which would look like below (from early rpm build):
   $ rpm -qpl kernel-tools-libs-5.0.0-1.fc31.x86_64.rpm
   /usr/lib/.build-id
   /usr/lib/.build-id/ca
   /usr/lib/.build-id/ca/654da1e5ea553f985e28b8d98ad24e51f19e88
   /usr/lib/.build-id/f6
   /usr/lib/.build-id/f6/a788b316f26fbe70db47bfc0ef500348117023
   /usr/lib64/libbpf.so.0
   /usr/lib64/libbpf.so.0.0.1
   /usr/lib64/libcpupower.so.0
   /usr/lib64/libcpupower.so.0.0.1
   /usr/share/licenses/kernel-tools-libs
   /usr/share/licenses/kernel-tools-libs/COPYING
   $ rpm -qpl kernel-tools-libs-devel-5.0.0-1.fc31.x86_64.rpm
   /usr/include/bpf/bpf.h
   /usr/include/bpf/btf.h
   /usr/include/bpf/libbpf.h
   /usr/include/cpufreq.h
   /usr/include/cpuidle.h
   /usr/lib64/libbpf.a
   /usr/lib64/libbpf.so
   /usr/lib64/libcpupower.so
Do you see libbpf as a standalone package or kernel-tools-libs* wuold be ok for you?

Quoted 7 lines

Quoted 4 lines
My preference is definitely on making libbpf a stand-alone package, so people can just install 'libbpf' or 'libbpf-dev{,el}' and are good to go. Also given the pace it's growing these days, it absolutely qualifies as a stand-alone package.
+1
libbpf should be standalone package and not part of kernel-tools.

Quoted 19 lines

Quoted 19 lines
Quoted 6 lines
2) There's already bcc-devel's libbpf library packaged:
   $ rpm -qf /usr/lib64/libbpf.so
   bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
   libbpf2.so
   libbpfobject.so
   libbpfbest.so
   ...?
I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.

Quoted 4 lines

Quoted 2 lines
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.
bcc migrated to libbpf some time ago.

yea, but looks like it still exports its own libbpf.so with some helpers:

[jolsa@krava ~]$ nm -D /usr/lib64/libbpf.so.0
...
0000000000005500 T bpf_attach_kprobe
0000000000005340 T bpf_attach_perf_event
00000000000052a0 T bpf_attach_perf_event_raw
0000000000004b50 T bpf_attach_raw_tracepoint
0000000000004af0 T bpf_attach_socket
0000000000005d10 T bpf_attach_tracepoint
0000000000005810 T bpf_attach_uprobe
0000000000004ea0 T bpf_attach_xdp
0000000000005480 T bpf_close_perf_event_fd
0000000000003990 T bpf_create_map
0000000000003cb0 T bpf_delete_elem
0000000000004b20 T bpf_detach_kprobe
0000000000004b40 T bpf_detach_tracepoint
0000000000004b30 T bpf_detach_uprobe
0000000000003d30 T bpf_get_first_key
0000000000003e70 T bpf_get_next_key
0000000000003c30 T bpf_lookup_elem
0000000000005fb0 T bpf_map_get_fd_by_id
0000000000005e40 T bpf_obj_get
0000000000003ef0 T bpf_obj_get_info
0000000000005dc0 T bpf_obj_pin
0000000000004c10 T bpf_open_perf_buffer
0000000000004d80 T bpf_open_perf_event
0000000000004980 T bpf_open_raw_sock
0000000000003f70 T bpf_prog_compute_tag
0000000000005f30 T bpf_prog_get_fd_by_id
0000000000005eb0 T bpf_prog_get_next_id
00000000000042f0 T bpf_prog_get_tag
0000000000004430 T bpf_prog_load
0000000000003ba0 T bpf_update_elem
00000000000061b0 T perf_reader_event_read
0000000000006520 T perf_reader_fd
0000000000006090 T perf_reader_free
0000000000006120 T perf_reader_mmap
0000000000006030 T perf_reader_new
00000000000063f0 T perf_reader_poll
0000000000006510 T perf_reader_set_fd
...

jirka

Michal Rostecki

2019-03-25 17:21 UTC #4 lore raw

In reply to Alexei Starovoitov

Quoted 18 lines · Alexei Starovoitov

Quoted 18 lines
Quoted 5 lines
2) There's already bcc-devel's libbpf library packaged:
   $ rpm -qf /usr/lib64/libbpf.so
   bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
   libbpf2.so
   libbpfobject.so
   libbpfbest.so
   ...?
I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.

Quoted 4 lines

Quoted 2 lines
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.
bcc migrated to libbpf some time ago.

And the libbpf.so file which is installed with bcc is "our" libbpf. bcc simply uses libbpf (from the auto-synced standalone repo[0]) as a submodule[1]. To package libbpf and bcc properly in Linux distros we need a possibility to use bcc with shared libbpf library - which probably can be achiveved by small change in bcc's CMakeLists.txt.

The standalone repo[0] might be a better base for creating a package than full kernel sources. And using %soversion%+git%sha% (i.e. 0.0.2+git33b017) might be a better version pattern of that package.

Cheers,
Michal

[0] https://github.com/libbpf/libbpf
[1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16

Yonghong Song

2019-03-25 19:35 UTC #5 lore raw

Quoted 22 lines · Michal Rostecki

Quoted 21 lines · Alexei Starovoitov
Quoted 20 lines
Quoted 5 lines
2) There's already bcc-devel's libbpf library packaged:
    $ rpm -qf /usr/lib64/libbpf.so
    bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
libbpf2.so
libbpfobject.so
libbpfbest.so
...?
I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.

Quoted 7 lines

Quoted 1 line
bcc migrated to libbpf some time ago.
And the libbpf.so file which is installed with bcc is "our" libbpf. bcc simply uses libbpf (from the auto-synced standalone repo[0]) as a submodule[1]. To package libbpf and bcc properly in Linux distros we need a possibility to use bcc with shared libbpf library - which probably can be achiveved by small change in bcc's CMakeLists.txt.

I think we should rename bcc libbpf.so to a different name (maybe libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo. The bcc libbpf.so is different from libbpf repo it contains some wrappers...

I will propose to iovisor/bcc mailing list.

Quoted 11 lines

The standalone repo[0] might be a better base for creating a package than full kernel sources. And using %soversion%+git%sha% (i.e. 0.0.2+git33b017) might be a better version pattern of that package.
Cheers,
Michal
[0] https://github.com/libbpf/libbpf
[1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16

Jiri Olsa

author

2019-03-25 20:20 UTC #6 lore raw

Quoted 33 lines · Yonghong Song

Quoted 22 lines · Michal Rostecki
Quoted 21 lines · Alexei Starovoitov
Quoted 20 lines
Quoted 5 lines
2) There's already bcc-devel's libbpf library packaged:
    $ rpm -qf /usr/lib64/libbpf.so
    bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
libbpf2.so
libbpfobject.so
libbpfbest.so
...?
I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.
Quoted 7 lines
Quoted 1 line
bcc migrated to libbpf some time ago.
And the libbpf.so file which is installed with bcc is "our" libbpf. bcc simply uses libbpf (from the auto-synced standalone repo[0]) as a submodule[1]. To package libbpf and bcc properly in Linux distros we need a possibility to use bcc with shared libbpf library - which probably can be achiveved by small change in bcc's CMakeLists.txt.

Quoted 3 lines

I think we should rename bcc libbpf.so to a different name (maybe libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo. The bcc libbpf.so is different from libbpf repo it contains some wrappers...

that'd be great first step.. then we could add libbpf package and make bcc depend on it as suggested by Michal

Quoted 2 lines

I will propose to iovisor/bcc mailing list.

please keep me in cc for that

thanks,
jirka

Quoted 12 lines

Quoted 12 lines
The standalone repo[0] might be a better base for creating a package than full kernel sources. And using %soversion%+git%sha% (i.e. 0.0.2+git33b017) might be a better version pattern of that package.
Cheers,
Michal
[0] https://github.com/libbpf/libbpf
[1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16

Yonghong Song

2019-03-27 05:31 UTC #7 lore raw

Jiri,

Quoted 33 lines

Quoted 29 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 5 lines
2) There's already bcc-devel's libbpf library packaged:
     $ rpm -qf /usr/lib64/libbpf.so
     bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
libbpf2.so
libbpfobject.so
libbpfbest.so
...?
I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.
Quoted 7 lines
Quoted 1 line
bcc migrated to libbpf some time ago.
And the libbpf.so file which is installed with bcc is "our" libbpf. bcc simply uses libbpf (from the auto-synced standalone repo[0]) as a submodule[1]. To package libbpf and bcc properly in Linux distros we need a possibility to use bcc with shared libbpf library - which probably can be achiveved by small change in bcc's CMakeLists.txt.
Quoted 3 lines
I think we should rename bcc libbpf.so to a different name (maybe libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo. The bcc libbpf.so is different from libbpf repo it contains some wrappers...

Quoted 7 lines

that'd be great first step.. then we could add libbpf package and make bcc depend on it as suggested by Michal
Quoted 2 lines
I will propose to iovisor/bcc mailing list.
please keep me in cc for that

Do not know your github name, so not able to cc you on the bcc pull request which changes library from libbpf.{a,so} to libbcc_bpf.{a,so}. The below is the link:

    https://github.com/iovisor/bcc/pull/2290

Quoted 16 lines

thanks,
jirka
Quoted 12 lines
Quoted 12 lines
The standalone repo[0] might be a better base for creating a package than full kernel sources. And using %soversion%+git%sha% (i.e. 0.0.2+git33b017) might be a better version pattern of that package.
Cheers,
Michal
[0] https://github.com/libbpf/libbpf
[1] https://github.com/iovisor/bcc/blob/master/CMakeLists.txt#L12-L16

Jiri Olsa

author

2019-03-27 07:20 UTC #8 lore raw

Quoted 39 lines · Yonghong Song

Jiri,
Quoted 37 lines
Quoted 29 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 5 lines
2) There's already bcc-devel's libbpf library packaged:
     $ rpm -qf /usr/lib64/libbpf.so
     bcc-devel-0.8.0-1.fc28.x86_64
Quoted 7 lines
so there's a conflict.. any chance we could rename libbpf to something else like:
libbpf2.so
libbpfobject.so
libbpfbest.so
...?
I don't think we should rename the official libbpf package, this will just create plain confusion and will make it much harder for potential users to adapt in the long-term since we aim for /everyone/ to consume official libbpf library instead of hacking their own.
I think bcc folks are migrating to official libbpf as well, at least that was my impression. Imho, this would need fixing on bcc side then.
Quoted 7 lines
Quoted 1 line
bcc migrated to libbpf some time ago.
And the libbpf.so file which is installed with bcc is "our" libbpf. bcc simply uses libbpf (from the auto-synced standalone repo[0]) as a submodule[1]. To package libbpf and bcc properly in Linux distros we need a possibility to use bcc with shared libbpf library - which probably can be achiveved by small change in bcc's CMakeLists.txt.
Quoted 3 lines
I think we should rename bcc libbpf.so to a different name (maybe libbcc_bpf.so) to avoid confusions between bcc libbpf and libbpf repo. The bcc libbpf.so is different from libbpf repo it contains some wrappers...
that'd be great first step.. then we could add libbpf package and make bcc depend on it as suggested by Michal
Quoted 1 line

Quoted 8 lines

Quoted 3 lines
Quoted 1 line
I will propose to iovisor/bcc mailing list.
please keep me in cc for that
Do not know your github name, so not able to cc you on the bcc pull request which changes library from libbpf.{a,so} to libbcc_bpf.{a,so}. The below is the link:
    https://github.com/iovisor/bcc/pull/2290

it's olsajiri, that should do it, thanks

jirka