tools lib traceevent: How to do library versioning being in the Linux kernel source?
First, I hope everyone had a Happy New Year!
Next, Sudip has been working to get the libtraceevent library into Debian. As this has been happening, I've been working at how to get all the projects that use this, to use the library installed on the system if it does exist. I'm hoping that once it's in Debian, the other distros will follow suit.
Currently, the home of libtraceevent lives in the Linux kernel source tree under tools/lib/traceevent. This was because perf uses it to parse the events, and it seemed logical (at the time) to use this location as the main source tree for the distributions.
The problem I'm now having is that I'm looking at fixing and updating some of the code in this library, and since library versioning is critical for applications that depend on it, we need to have a way to update the versions, and this does not correspond with the Linux versions.
For example, we currently have:
libtraceevent.so.1.1.0
If I make some bug fixes, I probably want to change it to:
libtraceevent.so.1.1.1 or libtraceevent.so.1.2.0
But if I change the API, which I plan on doing soon, I would probably need to update the major version.
libtraceevent.so.2.0.0
The thing is, we shouldn't be making these changes for every update that we send to the main kernel. I would like to have a minimum of tags to state what the version should be, and perhaps even branches for working on a development version.
This is a problem with living in the Linux source tree as tags and branches in Linus's tree are for only the Linux kernel source itself. This may work fine for perf, as it's not a library and there's not tools depending on the version of it. But it is a problem when it comes to shared libraries.
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it becomes something that is reliably in all distributions. It's not like perf doesn't depend on other libraries today anyway.
Thoughts or suggestions?
Thanks!
-- Steve
Em Thu, Jan 02, 2020 at 12:20:04PM -0500, Steven Rostedt escreveu:
Quoted 3 lines
This is a problem with living in the Linux source tree as tags and branches in Linus's tree are for only the Linux kernel source itself. This may work fine for perf, as it's not a library and there's not
[acme@quaco perf]$ git log --oneline tools/lib/perf 6364e2891a4c libperf: Add man pages 19e0154effb4 libperf: Move to tools/lib/perf [acme@quaco perf]$
[acme@quaco perf]$ git log --follow --oneline tools/lib/perf/Makefile 6364e2891a4c libperf: Add man pages19e0154effb4 libperf: Move to tools/lib/perf395e62cde10d libperf: Link static tests with libapi.a7728fa0cfaeb libperf: Adopt perf_mmap__consume() function from tools/perf fb4bf51fcc15 libperf: Add libperf dependency for tests targets d80a5540bccb libperf: Link libapi.a in libperf.so227cb129858a libperf: Add missing event.h file to install rule b81d39c7a1ef (tag: perf-core-for-mingo-5.4-20190820) libperf: Fix arch include paths 6a94b52a71b7 libperf: Add tests support0a64d7091efd libperf: Add install targets47f9bccc79cb libperf: Add build version support
314350491810 libperf: Make libperf.a part of the perf build [acme@quaco perf]$
libperf adopted the versioning and packaging practices introduced by tools/lib/bpf, perhaps you could do the same for tools/lib/traceevent and then we would have a standard for these cases?
The problem of having libperf, libbpf in distros is already being tackled for quite a while, would be interesting to follow what has happened in that area as well, Jiri knows more about this, Jiri?
- Arnaldo
[ Added the BPF folks ]
Quoted 10 lines · On Thu, 2 Jan 2020 15:42:52 -0300 Arnaldo Carvalho de Melo…
Em Thu, Jan 02, 2020 at 12:20:04PM -0500, Steven Rostedt escreveu:
Quoted 3 lines
This is a problem with living in the Linux source tree as tags and branches in Linus's tree are for only the Linux kernel source itself. This may work fine for perf, as it's not a library and there's not
libperf adopted the versioning and packaging practices introduced by
tools/lib/bpf, perhaps you could do the same fortools/lib/traceeventand then we would have a standard for these cases?
I don't see libperf in my Debian testing installation, but I did find libbpf.
Quoted 4 lines
The problem of having libperf, libbpf in distros is already being tackled for quite a while, would be interesting to follow what has happened in that area as well, Jiri knows more about this, Jiri?
Looking at the libbpf Makefile, it is getting the version from the libbpf.map file.
What's the standard way for distributions to know when to use the number? Only from Linux stable trees? That way, we can make fixes to the library, but still need to remember to update the version number before the release.
How does libbpf handle the version numbers with bug fixes? Does it get updated one a kernel release if there are changes?
Obviously, we need to update the major number if the API changes in anyway that is not compatible for a new application, which includes adding new functions. Right?
-- Steve
Quoted 2 lines · Steven Rostedt
First, I hope everyone had a Happy New Year!
Happy New Year to you too.
Quoted 6 lines
Next, Sudip has been working to get the libtraceevent library into Debian. As this has been happening, I've been working at how to get all the projects that use this, to use the library installed on the system if it does exist. I'm hoping that once it's in Debian, the other distros will follow suit.
I have sent you another patch for libtraceevent. And, assuming that you will not have any objection to that patch libtraceevent has been merged in Debian and is now available in Debian Sid releases. Thanks to Ben for all his suggestion and help.
The packages are at:
https://packages.debian.org/unstable/libtraceevent1
https://packages.debian.org/unstable/libtraceevent-dev
https://packages.debian.org/unstable/libtraceevent1-plugin
Signature · 2 lines
Regards Sudip
Quoted 18 lines · On Thu, 2 Jan 2020 22:43:34 +0000 Sudip Mukherjee <[email protected]>…
Quoted 2 lines · Steven Rostedt
First, I hope everyone had a Happy New Year!
Happy New Year to you too.
Quoted 6 lines
Next, Sudip has been working to get the libtraceevent library into Debian. As this has been happening, I've been working at how to get all the projects that use this, to use the library installed on the system if it does exist. I'm hoping that once it's in Debian, the other distros will follow suit.
I have sent you another patch for libtraceevent. And, assuming that you will not have any objection to that patch libtraceevent has been merged in Debian and is now available in Debian Sid releases. Thanks to Ben for all his suggestion and help.
Quoted 6 lines
The packages are at:
https://packages.debian.org/unstable/libtraceevent1
https://packages.debian.org/unstable/libtraceevent-dev
https://packages.debian.org/unstable/libtraceevent1-plugin
BTW, while doing some minor fixes, I realized I still have generic names for "warning", "pr_stat" and "vpr_stat" and thought they should be changed to "tep_warning", "tep_pr_stat" and "tep_vpr_stat" for namespace reasons even though they are weak functions. Would this require a major version change, or perhaps its early enough to get this in with a minor version change as the libraries are probably not being used yet?
-- Steve
Hello Steve,
Happy new year!
Quoted 7 lines · Steven Rostedt
BTW, while doing some minor fixes, I realized I still have generic names for "warning", "
pr_stat" and "vpr_stat" and thought they should be changed to "tep_warning", "tep_pr_stat" and "tep_vpr_stat" for namespace reasons even though they are weak functions. Would this require a major version change, or perhaps its early enough to get this in with a minor version change as the libraries are probably not being used yet?
Hmm.. I don't think it requires a major version change as they are not public APIs actually. They are internal functions but can be overridden by external ones, right?
I guess it's because there was a concern with printing messages in a library. What about adding a mechanism to register callbacks (and maybe defaults to discard all messages)?
Thanks
Namhyung
Quoted 1 line · Steven Rostedt
First, I hope everyone had a Happy New Year!
heya, Happy New Year! ;-)
Quoted 42 lines
Next, Sudip has been working to get the libtraceevent library into Debian. As this has been happening, I've been working at how to get all the projects that use this, to use the library installed on the system if it does exist. I'm hoping that once it's in Debian, the other distros will follow suit.
Currently, the home of libtraceevent lives in the Linux kernel source tree under
tools/lib/traceevent. This was because perf uses it to parse the events, and it seemed logical (at the time) to use this location as the main source tree for the distributions.The problem I'm now having is that I'm looking at fixing and updating some of the code in this library, and since library versioning is critical for applications that depend on it, we need to have a way to update the versions, and this does not correspond with the Linux versions.
For example, we currently have:
libtraceevent.so.1.1.0
If I make some bug fixes, I probably want to change it to:
libtraceevent.so.1.1.1 or libtraceevent.so.1.2.0
But if I change the API, which I plan on doing soon, I would probably need to update the major version.
libtraceevent.so.2.0.0
The thing is, we shouldn't be making these changes for every update that we send to the main kernel. I would like to have a minimum of tags to state what the version should be, and perhaps even branches for working on a development version.
This is a problem with living in the Linux source tree as tags and branches in Linus's tree are for only the Linux kernel source itself. This may work fine for perf, as it's not a library and there's not tools depending on the version of it. But it is a problem when it comes to shared libraries.
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is ready
libbpf github repo is then source for fedora (and others) package
Quoted 2 lines
becomes something that is reliably in all distributions. It's not like perf doesn't depend on other libraries today anyway.
yep, we already have a way to link libbpf dynamicaly from package, will work the same for libtraceevent
jirka
Quoted 10 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is readylibbpf github repo is then source for fedora (and others) package
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
We can have the tools/lib/traceevent be the main source, but then just copy it to the stand alone for releases.
Sudip, would this work for you too? (and yes, I plan on acking that patch for the -ldl change, after looking at it a little bit more).
-- Steve
Quoted 12 lines · Steven Rostedt
Quoted 8 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is ready
Quoted 5 lines
Quoted 2 lines
libbpf github repo is then source for fedora (and others) package
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
Debian is building libbpf from kernel tree.
Quoted 6 lines
We can have the
tools/lib/traceeventbe the main source, but then just copy it to the stand alone for releases.Sudip, would this work for you too? (and yes, I plan on acking that patch for the -ldl change, after looking at it a little bit more).
If it moves to a separate repo then I guess we will need to move it out of the debian kernel and create a separate package. Might be better if we do for libbpf also.
(Adding Ben).
Ben?
Signature · 2 lines
Regards Sudip
Quoted 11 lines · Steven Rostedt
Quoted 8 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is ready
Quoted 5 lines
Quoted 2 lines
libbpf github repo is then source for fedora (and others) package
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
sounds good, and if it works out, we'll follow you with libperf :-)
if you want to check on the libbpf:
https://github.com/libbpf/libbpf
there might be some syncs scripts worth checking
jirka
Quoted 9 lines
We can have the
tools/lib/traceeventbe the main source, but then just copy it to the stand alone for releases.Sudip, would this work for you too? (and yes, I plan on acking that patch for the -ldl change, after looking at it a little bit more).
-- Steve
Quoted 16 lines · Jiri Olsa
Quoted 14 lines · Steven Rostedt
Quoted 10 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is readylibbpf github repo is then source for fedora (and others) package
Quoted 7 lines
Quoted 2 lines
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
sounds good, and if it works out, we'll follow you with libperf :-)
if you want to check on the libbpf:
https://github.com/libbpf/libbpf
fwiw, I have opened a bug report in Debian requesting to package libbpf from the github repo.
Signature · 2 lines
Regards Sudip
[ Added Konstantin and kernel.org users mailing list ]
Quoted 18 lines · On Fri, 3 Jan 2020 14:36:40 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 8 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is readyQuoted 5 lines
Quoted 2 lines
libbpf github repo is then source for fedora (and others) package
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
Quoted 6 lines
sounds good, and if it works out, we'll follow you with libperf :-)
if you want to check on the libbpf:
https://github.com/libbpf/libbpfthere might be some syncs scripts worth checking
I wonder if there should be a:
git://git.kernel.org/pub/scm/utils/lib/
directory to have:
git://git.kernel.org/pub/scm/utils/lib/traceevent/
git://git.kernel.org/pub/scm/utils/lib/libbpf/
git://git.kernel.org/pub/scm/utils/lib/libperf/
That could hold the libraries that are tight to the kernel?
-- Steve
Quoted 12 lines
jirka
Quoted 9 lines
We can have the
tools/lib/traceeventbe the main source, but then just copy it to the stand alone for releases.Sudip, would this work for you too? (and yes, I plan on acking that patch for the -ldl change, after looking at it a little bit more).
-- Steve
Quoted 33 lines · Steven Rostedt
[ Added Konstantin and kernel.org users mailing list ]
Quoted 18 lines · On Fri, 3 Jan 2020 14:36:40 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 8 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is readyQuoted 5 lines
Quoted 2 lines
libbpf github repo is then source for fedora (and others) package
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
Quoted 6 lines
sounds good, and if it works out, we'll follow you with libperf :-)
if you want to check on the libbpf:
https://github.com/libbpf/libbpfthere might be some syncs scripts worth checking
I wonder if there should be a:
Quoted 7 lines
git://git.kernel.org/pub/scm/utils/lib/directory to have:
git://git.kernel.org/pub/scm/utils/lib/traceevent/
git://git.kernel.org/pub/scm/utils/lib/libbpf/
git://git.kernel.org/pub/scm/utils/lib/libperf/
we could loose the 'lib' and just have:
git://git.kernel.org/pub/scm/utils/lib/perf/
Quoted 2 lines
That could hold the libraries that are tight to the kernel?
I don't think libbpf will change now after they are settled in github, but we could consider this for libperf
jirka
Em Mon, Jan 06, 2020 at 04:19:02PM +0100, Jiri Olsa escreveu:
Quoted 38 lines · Steven Rostedt
Quoted 37 lines
[ Added Konstantin and kernel.org users mailing list ]
Quoted 18 lines · On Fri, 3 Jan 2020 14:36:40 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 11 lines · Steven Rostedt
Quoted 8 lines · On Fri, 3 Jan 2020 00:49:50 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 3 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it
so libbpf 'moved' for this reason to github repo,
but keeping the kernel as the true/first source,
and updating github repo when release is readyQuoted 5 lines
Quoted 2 lines
libbpf github repo is then source for fedora (and others) package
Ah, so perhaps I should follow this? I could keep it a kernel.org repo (as I rather have it there anyway).
Quoted 6 lines
sounds good, and if it works out, we'll follow you with libperf :-)
if you want to check on the libbpf:
https://github.com/libbpf/libbpfthere might be some syncs scripts worth checking
I wonder if there should be a:
git://git.kernel.org/pub/scm/utils/lib/directory to have:
Quoted 7 lines
Quoted 3 lines
git://git.kernel.org/pub/scm/utils/lib/traceevent/
git://git.kernel.org/pub/scm/utils/lib/libbpf/
git://git.kernel.org/pub/scm/utils/lib/libperf/we could loose the 'lib' and just have:
git://git.kernel.org/pub/scm/utils/lib/perf/
So, we have:
https://www.kernel.org/pub/linux/kernel/tools/perf/
trying to mimic the kernel sources tree structure, so perhaps we could have:
https://www.kernel.org/pub/linux/kernel/tools/lib/{perf,traceevent}/
To continue that directory tree mirror?
Quoted 6 lines
Quoted 1 line
That could hold the libraries that are tight to the kernel?
I don't think libbpf will change now after they are settled in github, but we could consider this for libperf
jirka
Signature · 1 line
- Arnaldo
Quoted 10 lines · On Mon, 6 Jan 2020 13:26:23 -0300 Arnaldo Carvalho de Melo…
So, we have:
https://www.kernel.org/pub/linux/kernel/tools/perf/
trying to mimic the kernel sources tree structure, so perhaps we could have:
https://www.kernel.org/pub/linux/kernel/tools/lib/{perf,traceevent}/
To continue that directory tree mirror?
Wouldn't that become a bit of manual work. Unlike perf, the versions will not correspond to the Linux kernel versions. They would need to follow library versioning.
It would at a minimum require new scripting to get this right.
-- Steve
Em Mon, Jan 06, 2020 at 11:36:15AM -0500, Steven Rostedt escreveu:
Quoted 10 lines
Quoted 6 lines · On Mon, 6 Jan 2020 13:26:23 -0300 Arnaldo Carvalho de Melo…
So, we have:
https://www.kernel.org/pub/linux/kernel/tools/perf/
trying to mimic the kernel sources tree structure, so perhaps we could have:
Quoted 7 lines
Quoted 3 lines
https://www.kernel.org/pub/linux/kernel/tools/lib/{perf,traceevent}/
To continue that directory tree mirror?
Wouldn't that become a bit of manual work. Unlike perf, the versions will not correspond to the Linux kernel versions. They would need to follow library versioning.
It doesn't have to correspond, the versions you use there are entirely up to libtraceevent developers, no? I.e. when you decide to cut some version, tag it in the linux kernel git repo, create a tarball, something like:
make help | grep perf
but using whatever versioning you decide to use, which would be the same regardless of where you develop it, and make it available via
https://www.kernel.org/pub/linux/kernel/tools/lib/traceevent/
Quoted 1 line
It would at a minimum require new scripting to get this right.
Sure, regardless of where you do source code control you will need to tag, create a tarball, signatures (which kup helps with) for kernel.org, for instance I use:
kup put perf-${VER}.tar.xz perf-${VER}.tar.sign /pub/linux/kernel/tools/perf/v${VER}/perf-${VER}.tar.xzWhat is in ${VER} is entirely up to me, its just that perf has a very active development process with lots of patches each release and we try to stop getting features when the kernel closes the window, have a -next for new features, etc, so we end up with perf-ver == kernel-ver, but that never was a requirement, its just convenient and we got used to it.
- Arnaldo
Quoted 5 lines · Arnaldo Carvalho de Melo
Sure, regardless of where you do source code control you will need to tag, create a tarball, signatures (which kup helps with) for kernel.org, for instance I use:
kup put perf-${VER}.tar.xz perf-${VER}.tar.sign /pub/linux/kernel/tools/perf/v${VER}/perf-${VER}.tar.xz
It's worth noting that you don't have to use kup if you don't want to -- we have a mechanism to create tarball releases directly from tag signatures. You just have to add a special note to the tag and the backend does the rest automatically -- we have a handy script [^1] to make it easier.
Greg KH has been using this process for a while now.
If you would like to switch to that instead of using kup directly, just let me know.
-K
[^1]: https://git.kernel.org/pub/scm/linux/kernel/git/mricon/korg-helpers.git/tree/git-archive-signer
Em Mon, Jan 06, 2020 at 03:14:01PM -0500, Konstantin Ryabitsev escreveu:
Quoted 13 lines · Arnaldo Carvalho de Melo
Quoted 5 lines
Sure, regardless of where you do source code control you will need to tag, create a tarball, signatures (which kup helps with) for kernel.org, for instance I use:
kup put perf-${VER}.tar.xz perf-${VER}.tar.sign /pub/linux/kernel/tools/perf/v${VER}/perf-${VER}.tar.xzIt's worth noting that you don't have to use kup if you don't want to -- we have a mechanism to create tarball releases directly from tag signatures. You just have to add a special note to the tag and the backend does the rest automatically -- we have a handy script [^1] to make it easier.
Quoted 4 lines
Greg KH has been using this process for a while now.
If you would like to switch to that instead of using kup directly, just let me know.
Sure I wanna know, will read [^1], thanks for the pointer.
On a side note since this is a library/tool that is hosted in the kernel, like perf, using that /pub/linux/kernel/tools/lib/ path seems appropriate, i.e. it provides tarballs for things in the tools/ directory of the linux/kernel/ :-)
- Arnaldo
Quoted 12 lines · Steven Rostedt
Quoted 9 lines · On Mon, 6 Jan 2020 13:26:23 -0300 Arnaldo Carvalho de Melo…
So, we have:
https://www.kernel.org/pub/linux/kernel/tools/perf/
trying to mimic the kernel sources tree structure, so perhaps we could have:
https://www.kernel.org/pub/linux/kernel/tools/lib/{perf,traceevent}/
Quoted 7 lines
Quoted 1 line
To continue that directory tree mirror?
Wouldn't that become a bit of manual work. Unlike perf, the versions will not correspond to the Linux kernel versions. They would need to follow library versioning.
It would at a minimum require new scripting to get this right.
If it is not tightly linked to the kernel and is just a normal library, you might consider using github. This is what we do for the rdma user space and it works well. We still take patches from the mailing list flow, but do use a fair amount of the github stuff too:
https://github.com/linux-rdma/rdma-core
With github actions now able to provide a quite good CI it covers a lot of required stuff for a library in one place, in a way that doesn't silo all the build infrastucture.
If you are interested in how we set it up I can write a longer email.
Regards,
Jason
Quoted 1 line · On Mon, 6 Jan 2020 16:47:15 -0400 Jason Gunthorpe <[email protected]> wrote:
If it is not tightly linked to the kernel and is just a normal
Well, it's used by perf, trace-cmd, power-top and rasdaemon (and perhaps even more). It lives in the kernel tree mainly because of perf.
Quoted 9 lines
library, you might consider using github. This is what we do for the rdma user space and it works well. We still take patches from the mailing list flow, but do use a fair amount of the github stuff too:
https://github.com/linux-rdma/rdma-core
With github actions now able to provide a quite good CI it covers a lot of required stuff for a library in one place, in a way that doesn't silo all the build infrastucture.
Github has ways to help with libraries? I'm totally clueless about this. I'm interested in hearing more.
Thanks,
-- Steve
Quoted 2 lines
If you are interested in how we set it up I can write a longer email.
Quoted 7 lines · Steven Rostedt
Quoted 1 line · On Mon, 6 Jan 2020 16:47:15 -0400 Jason Gunthorpe <[email protected]> wrote:
If it is not tightly linked to the kernel and is just a normal
Well, it's used by perf, trace-cmd, power-top and rasdaemon (and perhaps even more). It lives in the kernel tree mainly because of perf.
I see
Quoted 6 lines
Quoted 3 lines
With github actions now able to provide a quite good CI it covers a lot of required stuff for a library in one place, in a way that doesn't silo all the build infrastucture.
Github has ways to help with libraries? I'm totally clueless about this. I'm interested in hearing more.
These days it is a lot of work to get a library ready for the distributions and github now has a built-in CI (git hub actions) that lets projects run through all the build and in some cases runtime tests needed often and automatically.
For instance we build rdma-core for Centos 6,78, Fedora 31, Ubuntu, SuSe, cross compile on ARM and PPC, all automatically and all drive from a fairly short script in the source tree, so anyone can contribute.
The release process to .tar.gz (and distro packages if we wanted) is also automated via the same. Push a tag and all the release stuff is done and the right .tar.gz appears automatically in the right place.
It is nothing so unique, but having everything nicely integrated in one place makes it possible for a project to spend a small amount of time on CI and administration stuff instead of a large amount :)
Jason
Quoted 9 lines · Jiri Olsa
Quoted 9 lines
I wonder if there should be a:
git://git.kernel.org/pub/scm/utils/lib/directory to have:
git://git.kernel.org/pub/scm/utils/lib/traceevent/
git://git.kernel.org/pub/scm/utils/lib/libbpf/
git://git.kernel.org/pub/scm/utils/lib/libperf/
We already have 'pub/scm/libs' so I suggest we just place things there:
pub/scm/libs/traceevent
pub/scm/libs/libbpf
pub/scm/libs/libperf
Normally, we do a whole subdir, but that can be unnecessary, especially to avoid repetitions like:
pub/scm/libs/libgpiod/libgpiod.git
If the suggested locations work for you, just email [email protected] requesting their creation, and please specify who should have push access to them.
-K
Quoted 1 line · On Thu, 2 Jan 2020 12:20:04 -0500 Steven Rostedt <[email protected]> wrote:
First, I hope everyone had a Happy New Year!
Hope you to have a Happy New Year too!
Quoted 42 lines
Next, Sudip has been working to get the libtraceevent library into Debian. As this has been happening, I've been working at how to get all the projects that use this, to use the library installed on the system if it does exist. I'm hoping that once it's in Debian, the other distros will follow suit.
Currently, the home of libtraceevent lives in the Linux kernel source tree under
tools/lib/traceevent. This was because perf uses it to parse the events, and it seemed logical (at the time) to use this location as the main source tree for the distributions.The problem I'm now having is that I'm looking at fixing and updating some of the code in this library, and since library versioning is critical for applications that depend on it, we need to have a way to update the versions, and this does not correspond with the Linux versions.
For example, we currently have:
libtraceevent.so.1.1.0
If I make some bug fixes, I probably want to change it to:
libtraceevent.so.1.1.1 or libtraceevent.so.1.2.0
But if I change the API, which I plan on doing soon, I would probably need to update the major version.
libtraceevent.so.2.0.0
The thing is, we shouldn't be making these changes for every update that we send to the main kernel. I would like to have a minimum of tags to state what the version should be, and perhaps even branches for working on a development version.
This is a problem with living in the Linux source tree as tags and branches in Linus's tree are for only the Linux kernel source itself. This may work fine for perf, as it's not a library and there's not tools depending on the version of it. But it is a problem when it comes to shared libraries.
Quoted 5 lines
Should we move libtraceevent into a stand alone git repo (on kernel.org), that can have tags and branches specifically for it? We can keep a copy in the Linux source tree for perf to use till it becomes something that is reliably in all distributions. It's not like perf doesn't depend on other libraries today anyway.
+1. It sounds reasonable to move the main part of libtraceevent out of kernel tree. BTW, the plugins seems depending on the kernel. Maybe we can provide it under /lib/modules/<kversion>/shared/libtraceevent/plugins/ ? :)
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 4 lines · On Fri, 3 Jan 2020 21:17:43 +0900 Masami Hiramatsu <[email protected]> wrote:
+1. It sounds reasonable to move the main part of libtraceevent out of kernel tree. BTW, the plugins seems depending on the kernel. Maybe we can provide it under /lib/modules/<kversion>/shared/libtraceevent/plugins/ ? :)
They really shouldn't be. They should be able to be used between different versions of the kernel. Which plugins do you see an issue with?
-- Steve
Quoted 11 lines · On Fri, 3 Jan 2020 18:12:42 -0500 Steven Rostedt <[email protected]> wrote:
Quoted 4 lines · On Fri, 3 Jan 2020 21:17:43 +0900 Masami Hiramatsu <[email protected]> wrote:
+1. It sounds reasonable to move the main part of libtraceevent out of kernel tree. BTW, the plugins seems depending on the kernel. Maybe we can provide it under /lib/modules/<kversion>/shared/libtraceevent/plugins/ ? :)
They really shouldn't be. They should be able to be used between different versions of the kernel. Which plugins do you see an issue with?
I meant tools/lib/traceevent/plugins.
It seems that those plugins provide equivarent codes of some macros or values in event formats. In that case, when a new kernel modifies the event definitions, I think some of those needs to be updated too. Or, would those events be considered as a stable ABI?
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>