tracing/events: stable tracepoints
Subsystems (3)
Steven Rostedt
maintainercover tracing/events: stable tracepoints
[ RFC ONLY - Not for inclusion ]
As discussed at Kernel Summit, there was some issues about what to do with tracepoints.
Basically, anyone, anywhere, any developer, can create a tracepoint and have it appear in /sys/kernel/debug/tracing/events/...
These events automatically appear in both perf and ftrace as events. And any tool can tap into them. That's where the problem rises.
What happens when a tool starts to depend on a tracepoint? Will that tracepoint always be there? Will it ever change?
The problem also extends to the fact that we can't guarantee that tracepoints will stay as is. There are literally hundreds of tracepoints, and they are used by developers to have in field debugging tools. As the kernel changes, so will these tracepoints. A developer can use these to ask a customer that has run into some problem to enable a trace and send the developer back the trace so they can go off and analyze it.
But for tools, this is a different story. They want and depend on a tracepoint to be stable. If it changes under them, then it makes tracepoints completely useless for tools.
This patch series is a start and RFC for the creation of
stable tracepoints. I will now call the current tracepoints raw or in-field-debugging tracepoints or events. What I call stable tracepoints are those that are to answer questions about the OS and not for a developer to debug their code.
What I propose is to create a new format and a new filesystem called eventfs. Like debugfs, when enabled, a directory will be created:
/sys/kernel/events
Which would be the normal place to mount the eventfs filesystem.
The old format for events looked like this:
$ cat /debug/tracing/events/sched/sched_switch/format name: sched_switch ID: 57 format: field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1; field:int common_lock_depth; offset:8; size:4; signed:1;
field:char prev_comm[TASK_COMM_LEN]; offset:12; size:16; signed:1; field:pid_t prev_pid; offset:28; size:4; signed:1; field:int prev_prio; offset:32; size:4; signed:1; field:long prev_state; offset:40; size:8; signed:1; field:char next_comm[TASK_COMM_LEN]; offset:48; size:16; signed:1; field:pid_t next_pid; offset:64; size:4; signed:1; field:int next_prio; offset:68; size:4; signed:1;
print fmt: "prev_comm=%s prev_pid=%d prev_prio=%d prev_state=%s ==> next_comm=%s next_pid=%d next_prio=%d", REC->prev_comm, REC->prev_pid, REC->prev_prio, REC->prev_state ? __print_flags(REC->prev_state, "|", { 1, "S"} , { 2, "D" }, { 4, "T" }, { 8, "t" }, { 16, "Z" }, { 32, "X" }, { 64, "x" }, { 128, "W" }) : "R", REC->next_comm, REC->next_pid, REC->next_prio
The "common" fields were ftrace (and because perf attached to it, also perf) specific. Also the size is in bytes, which would limit the ability to use bit fields. We also don't know about arch specific alignment that may be needed to write to these fields.
We also have name (redundant), ID (should be agnostic), and print_fmt (lots of issues).
So the new format looks like this:
[root@bxf ~]# cat /sys/kernel/event/sched_switch/format
array:prev_comm type:char size:8 count:16 align:1 signed:1; field:prev_pid type:pid_t size:32 align:4 signed:1; field:prev_state type:char size:8 align:1 signed:1; array:next_comm type:char size:8 count:16 align:1 signed:1; field:next_pid type:pid_t size:32 align:4 signed:1;
Some notes:
o The size is in bits.
o We added an align, that is the natural alignment for the arch of that type.
o We added an "array" type, that specifies the size of an element as well as a "count", where total size can be align(size) * count.
o We separated the field name from the type.
Not in this series, but for future (after we agree on all this) I would like to move the raw tracepoints into /debug/events/... and have the same format as here.
This patch series uses some of the same tricks as the TRACE_EVENT() code. It has magic macros to do all the redundant code. But it has a bit of manual work.
Right now, when a STABLE_EVENT() is created, the format appears. But nothing hooks into it yet. perf, trace, or ftrace could register a handle that is created, either manually, or it can use the same magic macro tricks to automate all the stable events. The design has been made to allow for that too.
The last two patches create two stable tracepoints. sched_switch and sched_migrate_task (for examples as well as to get the ball rolling). As you may have already noticed, there is currently no hierarchy with the stable events. We want to limit the # of stable events, as they should only be created to help answer general questions about the OS. All events reside at the top layer of the eventfs filesystem. (I do not plan on doing this for the raw events though).
Another note is that all stable events need a corresponding raw event. The raw event does not need to be of the same format as the stable event, it just needs to provide all the information that the stable event needs, but the raw event may supply much more. This should not be a problem, since the tracepoint that represents a stable event should, by definition, always be stable :-)
Because the stable events piggy back on top of the raw events, the trace_...() function in the kernel can be used by both. No changes are needed there. As long as there's already a tracepoint represented by a raw event, a stable event can be placed on top.
The raw event may change at anytime, as long as it always supplies the stable event with what is needed. It will require the hooks between them to be updated. The way tracepoints work, if they become out of sync, the code will fail to compile.
Time to get out the hose!
-- Steve
The following patches are in:
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
branch: rfc/events
Steven Rostedt (5):
events: Add EVENT_FS the event filesystem tracing/events: Add code to (un)register stable events tracing/events: Add infrastructure to show stable event formats tracing/events: Add stable event sched_switch tracing/events: Add sched_migrate_task stable event
----
fs/Kconfig | 6 + fs/Makefile | 1 + fs/eventfs/Makefile | 4 + fs/eventfs/file.c | 53 +++++ fs/eventfs/inode.c | 433 ++++++++++++++++++++++++++++++++++++++++++ include/linux/eventfs.h | 83 ++++++++ include/linux/magic.h | 3 +- include/trace/stable.h | 72 +++++++ include/trace/stable/sched.h | 33 ++++ include/trace/stable_list.h | 3 + kernel/Makefile | 1 + kernel/events/Makefile | 1 + kernel/events/event_format.c | 74 +++++++ kernel/events/event_format.h | 64 ++++++ kernel/events/event_reg.h | 79 ++++++++ kernel/events/events.c | 48 +++++ kernel/trace/Kconfig | 1 + 17 files changed, 958 insertions(+), 1 deletions(-)
Steven Rostedt
maintainerRFC PATCH 1/5 events: Add EVENT_FS the event filesystem
From: Steven Rostedt <[email protected]>
Copied mostly from debugfs, the eventfs is the filesystem that will include stable tracepoints. Currently nothing enables this filesystem as of this patch.
--- fs/Kconfig | 6 + fs/Makefile | 1 + fs/eventfs/Makefile | 4 + fs/eventfs/file.c | 53 ++++++ fs/eventfs/inode.c | 433 +++++++++++++++++++++++++++++++++++++++++++++++ include/linux/eventfs.h | 83 +++++++++ include/linux/magic.h | 3 +- 7 files changed, 582 insertions(+), 1 deletions(-) create mode 100644 fs/eventfs/Makefile create mode 100644 fs/eventfs/file.c create mode 100644 fs/eventfs/inode.c create mode 100644 include/linux/eventfs.h
fs/Kconfig +6 -0
@@ -250,6 +250,12 @@ source "fs/partitions/Kconfig" endmenu endif +config EVENT_FS + bool + help + The event filesystem, usually mounted at /sys/kernel/events. + This option is selected by other configs that require it. + source "fs/nls/Kconfig" source "fs/dlm/Kconfig"
fs/Makefile +1 -0
@@ -116,6 +116,7 @@ obj-$(CONFIG_HOSTFS) += hostfs/ obj-$(CONFIG_HPPFS) += hppfs/ obj-$(CONFIG_CACHEFILES) += cachefiles/ obj-$(CONFIG_DEBUG_FS) += debugfs/ +obj-$(CONFIG_EVENT_FS) += eventfs/ obj-$(CONFIG_OCFS2_FS) += ocfs2/ obj-$(CONFIG_BTRFS_FS) += btrfs/ obj-$(CONFIG_GFS2_FS) += gfs2/
fs/eventfs/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +eventfs-objs := inode.o file.o + +obj-$(CONFIG_EVENT_FS) += eventfs.o +
fs/eventfs/file.c new file +53 -0
53 lines not on this page · raw message
fs/eventfs/inode.c new file +433 -0
433 lines not on this page · raw message
include/linux/eventfs.h new file +83 -0
83 lines not on this page · raw message
include/linux/magic.h +2 -1
@@ -8,7 +8,8 @@#define CODA_SUPER_MAGIC 0x73757245#define CRAMFS_MAGIC 0x28cd3d45/* some random number */ #define CRAMFS_MAGIC_WEND 0x453dcd28/* magic number with the wrong endianess */ -#define DEBUGFS_MAGIC 0x64626720 +#define DEBUGFS_MAGIC 0x64626720 +#define EVENTFS_MAGIC 0x65667310#define SYSFS_MAGIC 0x62656572#define SECURITYFS_MAGIC 0x73636673#define SELINUX_MAGIC 0xf97cff8c
Signature · 1 line
1.7.1
Re: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 5 lines · Steven Rostedt
From: Steven Rostedt <[email protected]>
Copied mostly from debugfs, the eventfs is the filesystem that will include stable tracepoints. Currently nothing enables this filesystem as of this patch.
What? Wait, I wrote tracefs a long time ago just for this, why not take that code and use it instead?
{sigh} And I just deleted that old tracefs git tree today as I thought that idea was long gone and dead....
Quoted 9 lines
Cc Greg KH <[email protected]>Signed-off-by Steven Rostedt <[email protected]>--- fs/Kconfig | 6 + fs/Makefile | 1 + fs/eventfs/Makefile | 4 + fs/eventfs/file.c | 53 ++++++ fs/eventfs/inode.c | 433 +++++++++++++++++++++++++++++++++++++++++++++++
This seems a bit big, I don't think you need all of this for some reason. Are you sure you can't make it smaller?
thanks,
greg k-h
Re: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 8 lines · Greg KH
Quoted 4 lines · Steven Rostedt
From: Steven Rostedt <[email protected]>
Copied mostly from debugfs, the eventfs is the filesystem that will include stable tracepoints. Currently nothing enables this filesystem as of this patch.
What? Wait, I wrote tracefs a long time ago just for this, why not take that code and use it instead?
Yeah, and i know that i suggested 'eventfs' to Steve and others in a prior thread a few months ago - and i suspect Steve was following up on that suggestion with this patch? So i guess it's partly my fault ;-)
[ Also, i think our _real_ problems with tracing lie entirely elsewhere, but i've
explained that numerous times. Maintaining instrumentation bits is the ultimate cat herding experience ;-) ]
I also explained it in that eventfs suggestion thread that eventfs (or, indeed tracefs) is IMO only a second tier approach compared to the real thing: proper enumeration of events in sysfs.
[ Beyond the obvious compatibility detail that we are _NOT_ getting rid of
/debug/tracing/events/, as existing tooling depends on it. So unless eventfs or sysfs integration brings some real tangible benefits over what we have already we dont want to force tooling to migrate to yet another API. ]
Lin Ming and PeterZ are working on sysfs integration and they have posted several iterations of that work which extends event details to sysfs. That work is not complete yet and they need help. (I've Cc:-ed them.)
The sysfs approach has numerous upsides:
- Design: sysfs is a mature, multi-year project with tons of meaningful hardware and software hieararchies already well established. Attaching events to these existing nodes optionally is an obvious advantage and avoids duplication and forces people to think about structure.
- Concentration of structure: subsystem and driver authors/maintainers already care about their sysfs layout - and when they define new tracepoints for subsystem or driver instrumentation it would be very natural for those events to go somewhere nearby, in the existing sysfs hieararchy.
- Practicalities: sysfs is already mounted on all distros so tooling could rely on it universally. It's the ultimate 'describe system structure' store.
- Long term maintenance: we want to be strict with events, i.e. keep the descriptors read only and single-line structured. You sysfs folks are enforcing that pretty well - with eventfs we'd always have the nasty lure to apply API hacks to eventfs components when we really shouldnt ...
Eventfs has a couple of downsides:
- Design: it's slapping events into a separate, partly duplicated, partly unique, partly inconsistent set of hierarchies. We can deal with it, but it's not particularly intelligent and i'd like us to try harder.
- Practicalities: eventfs has to be mounted on every distro. It's an uphill climb in general and the appeal of an approach has to be _strong_ for this to be feasible.
So putting it into sysfs looks like a pretty intelligent solution all around and i'd prefer it.
Steve, would you be interested in helping out Lin Ming and PeterZ with the sysfs work - or at least help them come to the conclusion that we want eventfs?
Thanks,
Ingo
Steven Rostedt
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 8 lines · Ingo Molnar
Quoted 6 lines · Greg KH
Quoted 4 lines · Steven Rostedt
From: Steven Rostedt <[email protected]>
Copied mostly from debugfs, the eventfs is the filesystem that will include stable tracepoints. Currently nothing enables this filesystem as of this patch.
Quoted 6 lines
Quoted 2 lines
What? Wait, I wrote tracefs a long time ago just for this, why not take that code and use it instead?
Yeah, and i know that i suggested 'eventfs' to Steve and others in a prior thread a few months ago - and i suspect Steve was following up on that suggestion with this patch? So i guess it's partly my fault ;-)
And we brought this up at Kernel Summit.
Quoted 5 lines
[ Also, i think our _real_ problems with tracing lie entirely elsewhere, but i've
explained that numerous times. Maintaining instrumentation bits is the ultimate cat herding experience ;-) ]
Quoted 8 lines
I also explained it in that eventfs suggestion thread that eventfs (or, indeed tracefs) is IMO only a second tier approach compared to the real thing: proper enumeration of events in sysfs.
[ Beyond the obvious compatibility detail that we are _NOT_ getting rid of
/debug/tracing/events/, as existing tooling depends on it. So unless eventfs or sysfs integration brings some real tangible benefits over what we have already we dont want to force tooling to migrate to yet another API. ]
One benefit is that we have a way to distinguish between in-field-debugging tracepoints and tracepoints that are only for analysis tools.
Quoted 4 lines
Lin Ming and PeterZ are working on sysfs integration and they have posted several iterations of that work which extends event details to sysfs. That work is not complete yet and they need help. (I've Cc:-ed them.)
Actually, I suck at adding anything to the sysfs/kobject code. I always screw it up. I only got the /sys/kernel/events working because I copied it directly from Greg. I doubt I'd be much help.
Quoted 13 lines
The sysfs approach has numerous upsides:
- Design: sysfs is a mature, multi-year project with tons of meaningful hardware and software hieararchies already well established. Attaching events to these existing nodes optionally is an obvious advantage and avoids duplication and forces people to think about structure.
- Concentration of structure: subsystem and driver authors/maintainers already care about their sysfs layout - and when they define new tracepoints for subsystem or driver instrumentation it would be very natural for those events to go somewhere nearby, in the existing sysfs hieararchy.
Quoted 7 lines
- Practicalities: sysfs is already mounted on all distros so tooling could rely on it universally. It's the ultimate 'describe system structure' store.
- Long term maintenance: we want to be strict with events, i.e. keep the descriptors read only and single-line structured. You sysfs folks are enforcing that pretty well - with eventfs we'd always have the nasty lure to apply API hacks to eventfs components when we really shouldnt ...
Are these events now going to be labeled as stable? Is every tracepoint we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.
Also, if we just blindly label a tracepoint as "stable" then we must keep all its contents. For example, the sched_switch will contain the priority. As Peter has stated several times, that may go away. We also do not want to lose getting that information, as a lot of us use it.
Quoted 10 lines
Eventfs has a couple of downsides:
- Design: it's slapping events into a separate, partly duplicated, partly unique, partly inconsistent set of hierarchies. We can deal with it, but it's not particularly intelligent and i'd like us to try harder.
- Practicalities: eventfs has to be mounted on every distro. It's an uphill climb in general and the appeal of an approach has to be _strong_ for this to be feasible.
Some distros already mount debugfs by default. It's a oneliner in fstab.
Quoted 3 lines
So putting it into sysfs looks like a pretty intelligent solution all around and i'd prefer it.
Another downside is that you need to scan hundreds of directories to find tracepoints. And again, are they all now stable?
Quoted 3 lines
Steve, would you be interested in helping out Lin Ming and PeterZ with the sysfs work - or at least help them come to the conclusion that we want eventfs?
I don't think I would be much help with the former, and I'm thinking I'm losing the later.
Hmm, seems that every decision that we came to agreement with at Kernel Summit has been declined in practice. Makes me think that Kernel Summit is pointless, and was a waste of my time. :-(
-- Steve
Quoted 4 lines
Thanks,
Ingo
Re: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 3 lines · Steven Rostedt
Are these events now going to be labeled as stable? Is every tracepoint we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.
I think you are worrying about the wrong things.
I think Arjan's complaints at the KS stemmed from prior sporadic declarations on lkml that there is no tracepoint ABI _at all_, and that powertop/latencytop could break anytime.
But in reality i strongly disagree with such declarations, and tracepoint data that is used by PowerTop/timechart/latencytop or perf is and was an ABI, simple as that - and i've been enforcing that for two years. (We have so few good instrumentation tools that we _really_ dont want to break them.)
At that point, realizing that we have an ABI for existing tools, i think it's fundamentally misguided to go out on a limb trying to put barriers in the way of other tools that do not even exist to begin with ...
Our real problem with tracing is lack of relevance, lack of utility, lack of punch-through analytical power.
Trying to create a sandbox to _reduce utility_ is like the last step, and a really optional step, when we have such variety that we want some control over it. It's always expensive, it always reduces the tool space as collateral damage.
So please dont think of sysfs or eventfs as a tool to restrict. Think of it as a tool to _organize_.
Again, i'd _LOVE_ to have the 'problem' of us having so many tools that analyze application and kernel behavior in such a rich way that they use tracepoints that were not supposed to be 'stable'.
I simply dont see the 'problem' that is being solved here. We had a stable ABI and we didnt break sysprof or powertop/latencytop in the past and wont break it in the future either.
Thanks,
Ingo
Peter Zijlstra
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 3 lines · Ingo Molnar
I think Arjan's complaints at the KS stemmed from prior sporadic declarations on lkml that there is no tracepoint ABI
_atall_, and that powertop/latencytop could break anytime.
And it will, afaik Arjan refused to even parse the format file which is part of the tracepoint abi and I'll be changing those for the scheduler.
I really object to not being able to make sane changes just because some tool is too lazy to even implement the full ABI that was exposed.
Quoted 3 lines
I think Arjan's complaints at the KS stemmed from prior sporadic declarations on lkml that there is no tracepoint ABI
_atall_, and that powertop/latencytop could break anytime.
I fully intent to break powertop/latencytop if they refuse to use the format file, deal with it.
Also, in the unlikely event we need to re-order the task->state bits I'll do so without a moments hesitation, regardless of who consumes them through the scheduler tracepoints, that's simply not stuff that should be tied down.
The same for anything that tries to interpret task->prio through the tracepoints.
Steven Rostedt
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
In reply to Ingo MolnarQuoted 31 lines · Ingo Molnar
Quoted 3 lines · Steven Rostedt
Are these events now going to be labeled as stable? Is every tracepoint we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.
I think you are worrying about the wrong things.
I think Arjan's complaints at the KS stemmed from prior sporadic declarations on lkml that there is no tracepoint ABI
_atall_, and that powertop/latencytop could break anytime.But in reality i strongly disagree with such declarations, and tracepoint data that is used by PowerTop/timechart/latencytop or perf is and was an ABI, simple as that - and i've been enforcing that for two years. (We have so few good instrumentation tools that we _really_ dont want to break them.)
At that point, realizing that we have an ABI for existing tools, i think it's fundamentally misguided to go out on a limb trying to put barriers in the way of other tools that do not even exist to begin with ...
Our real problem with tracing is lack of relevance, lack of utility, lack of punch-through analytical power.
Trying to create a sandbox to
_reduceutility_is like the last step, and a really optional step, when we have such variety that we want some control over it. It's always expensive, it always reduces the tool space as collateral damage.So please dont think of sysfs or eventfs as a tool to restrict. Think of it as a tool to _organize_.
Quoted 7 lines
Again, i'd _LOVE_ to have the 'problem' of us having so many tools that analyze application and kernel behavior in such a rich way that they use tracepoints that were not supposed to be 'stable'.
I simply dont see the 'problem' that is being solved here. We had a stable ABI and we didnt break sysprof or powertop/latencytop in the past and wont break it in the future either.
What about a tool that picks up tracepoints that were only used by a developer for in-field debugging, and then that tracepoint disappears because of a design change. Is it OK for that tool to break with it?
Do all tools that use tracepoints require a "check" feature?
I guess the problem is that creators of the tools to analyze the kernel have no idea of what they can count on and what they can't. Do we need a process to have these tool creators request to developers to "keep this tracepoint"?
-- Steve
Peter Zijlstra
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 5 lines · Steven Rostedt
What about a tool that picks up tracepoints that were only used by a developer for in-field debugging, and then that tracepoint disappears because of a design change. Is it OK for that tool to break with it?
Do all tools that use tracepoints require a "check" feature?
Not sure what you mean with a 'check' feature, but I do think its useful to tools authors to clearly delineate between stable and debug tracepoints, that also facilitates
Quoted 4 lines
I guess the problem is that creators of the tools to analyze the kernel have no idea of what they can count on and what they can't. Do we need a process to have these tool creators request to developers to "keep this tracepoint"?
the process mentioned here, if they cannot find the data they want through existing stable tracepoints they have to request it, or otherwise clearly suffer breakage whenever we feel like changing stuff.
A nice aspect of devs coming to us and requesting data is that we get a fairly good idea of what tools are available and get to discuss the problem they're trying to solve.
Esp. that latter point is a very good one imho, we might have a totally different view on some of the problems :-)
Re: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
In reply to Ingo MolnarQuoted 10 lines · Ingo Molnar
I think Arjan's complaints at the KS stemmed from prior sporadic declarations on lkml that there is no tracepoint ABI
_atall_, and that powertop/latencytop could break anytime.But in reality i strongly disagree with such declarations, and tracepoint data that is used by PowerTop/timechart/latencytop or perf is and was an ABI, simple as that - and i've been enforcing that for two years. (We have so few good instrumentation tools that we _really_ dont want to break them.)
There was general consensus at the kernel summit that the tracepoints, being deliberately in /sys/kernel/debug was unstable and today exposes internal implementation details --- and that that people putting different traceponts in Documentation/ABI/{testing,stable,unstable} simply doesn't work.
Heck, I've recently had to make changes to tracepoints because otherwise perf would fall over dead when it tripped over an ext4 tracepoint, due to its limitations dealing with what we could put into TP_PRINTK(). Saying that tracepoints are stable ABI that must never change, forever and ever, amen, is simply a non-starter.
Quoted 3 lines
I simply dont see the 'problem' that is being solved here. We had a stable ABI and we didnt break sysprof or powertop/latencytop in the past and wont break it in the future either.
I think the general consensus is that we didn't have a stable ABI, and the tension between what kernel developers need for debugging, which of necessity is kernel version specific, and what gets exposed as a stable ABI that tools can depend upon, is not one that can be resolved.
Regards,
- Ted
Peter Zijlstra
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
In reply to Steven RostedtQuoted 6 lines · Steven Rostedt
Quoted 4 lines · Ingo Molnar
I also explained it in that eventfs suggestion thread that eventfs (or, indeed tracefs) is IMO only a second tier approach compared to the real thing: proper enumeration of events in sysfs.
Quoted 8 lines
Quoted 4 lines
[ Beyond the obvious compatibility detail that we are _NOT_ getting rid of
/debug/tracing/events/, as existing tooling depends on it. So unless eventfs or sysfs integration brings some real tangible benefits over what we have already we dont want to force tooling to migrate to yet another API. ]One benefit is that we have a way to distinguish between in-field-debugging tracepoints and tracepoints that are only for analysis tools.
Right, and there was a clear consensus that that was a desired thing, have a strong signal towards tool authors to request 'stable' tracepoints or live with the fact they have to cope with the thing changing.
As to the whole /debug/tracing/events thing, we can deprecate it like we have done so many interfaces, provided we have indeed provided an alternative interface.
Quoted 7 lines
Quoted 3 lines
Lin Ming and PeterZ are working on sysfs integration and they have posted several iterations of that work which extends event details to sysfs. That work is not complete yet and they need help. (I've Cc:-ed them.)
Actually, I suck at adding anything to the sysfs/kobject code. I always screw it up. I only got the
/sys/kernel/eventsworking because I copied it directly from Greg. I doubt I'd be much help.
Yeah, I suck at it too, sysfs is scary. Luckily Kay is helping me out.
Quoted 1 line
Are these events now going to be labeled as stable?
For now I'm concentrating on the hardware events, those are more or less stable in that if you run it on the same hardware you get the same thing counted. Different hardware may miss some events since it simply cannot provide anything resembling, or count something related but slightly different (speculative vs retired events, or different cache level, or a slightly different definition of miss etc..)
Basically the same we already have with the perf 'generic' events.
Quoted 8 lines
Is every tracepoint
we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.Also, if we just blindly label a tracepoint as "stable" then we must keep all its contents. For example, the
sched_switchwill contain the priority. As Peter has stated several times, that may go away. We also do not want to lose getting that information, as a lot of us use it.
For now I've not at all looked at representing tracepoints in sysfs, for the hardware bits I'm looking to place the event_source (formerly known as PMU) in the hardware topology already present in /sys/devices/.
One suggestion was to place the software and tracepoint events in /sys/kernel/ some place. Another was to place driver specific, say wifi things near the wifi driver node.
None of the proposals have dealt with the stable vs debug thing, simply because none of them were post KS.
Quoted 1 line
Some distros already mount debugfs by default. It's a oneliner in fstab.
I haven't yet seen that.. maybe its automounted on /sys/kernel/debug/ or some daft place like that by magic initscipts outside of fstab but I'd not notice that.
Quoted 6 lines
Quoted 3 lines
So putting it into sysfs looks like a pretty intelligent solution all around and i'd prefer it.
Another downside is that you need to scan hundreds of directories to find tracepoints. And again, are they all now stable?
Not really, they'd be accessible through the bus structure, something like:
/sys/bus/event_source/*/events/*
Sure, that's more than 1 directory, but then so
is /debug/tracing/events/*/*/
Quoted 5 lines
Quoted 2 lines
Steve, would you be interested in helping out Lin Ming and PeterZ with the sysfs work - or at least help them come to the conclusion that we want eventfs?
I don't think I would be much help with the former, and I'm thinking I'm losing the later.
Yeah, eventfs simply won't work for what we want to do with hardware events.
Quoted 3 lines
Hmm, seems that every decision that we came to agreement with at Kernel Summit has been declined in practice. Makes me think that Kernel Summit is pointless, and was a waste of my time. :-(
Well, I don't know, clearly Ingo seems to disagree, but then he wasn't at KS. Thomas and me were, and neither of us really see a problem with the stable vs debug things (at least, I didn't hear tglx protest).
Steven Rostedt
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Not sure if the stable vs debug tracepoints debate is dead.
Quoted 21 lines · Peter Zijlstra
Quoted 2 lines
Are these events now going to be labeled as stable?
For now I'm concentrating on the hardware events, those are more or less stable in that if you run it on the same hardware you get the same thing counted. Different hardware may miss some events since it simply cannot provide anything resembling, or count something related but slightly different (speculative vs retired events, or different cache level, or a slightly different definition of miss etc..)
Basically the same we already have with the perf 'generic' events.
Quoted 8 lines
Is every tracepoint
we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.Also, if we just blindly label a tracepoint as "stable" then we must keep all its contents. For example, the
sched_switchwill contain the priority. As Peter has stated several times, that may go away. We also do not want to lose getting that information, as a lot of us use it.
Quoted 7 lines
For now I've not at all looked at representing tracepoints in sysfs, for the hardware bits I'm looking to place the
event_source(formerly known as PMU) in the hardware topology already present in/sys/devices/.One suggestion was to place the software and tracepoint events in
/sys/kernel/some place. Another was to place driver specific, say wifi things near the wifi driver node.
Should we create a /sys/kernel/tracepoints/
directory for the stable tracepoints. And then have these directories have the format of those tracepoints? Or is that against the "sysfs" rule of only having a single line per file? The format is multi lines.
Quoted 10 lines
None of the proposals have dealt with the stable vs debug thing, simply because none of them were post KS.
Quoted 1 line
Some distros already mount debugfs by default. It's a oneliner in fstab.
I haven't yet seen that.. maybe its automounted on
/sys/kernel/debug/or some daft place like that by magic initscipts outside of fstab but I'd not notice that.
I'd still like to keep the general tracepoints in something like /sys/kernel/debug/events/... using the same format that we come up with for stable tracepoints. The fact that you need to mount the debugfs system to use it, should help keep some tools from using it.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
So putting it into sysfs looks like a pretty intelligent solution all around and i'd prefer it.
Quoted 7 lines
Quoted 2 lines
Another downside is that you need to scan hundreds of directories to find tracepoints. And again, are they all now stable?
Not really, they'd be accessible through the bus structure, something like:
/sys/bus/event_source/*/events/*
That's for every event? Or just software ones?
Quoted 5 lines
Sure, that's more than 1 directory, but then so
is /debug/tracing/events/*/*/
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Steve, would you be interested in helping out Lin Ming and PeterZ with the sysfs work - or at least help them come to the conclusion that we want eventfs?
I don't think I would be much help with the former, and I'm thinking I'm losing the later.
Yeah, eventfs simply won't work for what we want to do with hardware events.
Well, hardware events are something that only depends on what you have for hardware. I wasn't thinking of putting them in with the software events. They probably should be separate.
Can the hardware events be traced? That is, not just profiled, but have them traced for when they occur individually.
I'll go work on other things until we can come up with an agreement. I hate to keep wasting days of work just to have someone NAK it again.
-- Steve
Mathieu Desnoyers
reviewerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
In reply to Steven Rostedt* Steven Rostedt ([email protected]) wrote:
[...]
Quoted 3 lines
Are these events now going to be labeled as stable? Is every tracepoint we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.
I'd like to bring up the point of KVM tracepoints here. KVM can be configured as a module, and may clearly contain tracepoints that we'd like to be stable.
My thought is that what we really want to enforce is "no stable tracepoints in drivers" rather than in "modules", but I might be wrong.
Thoughts ?
Mathieu
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com
Steven Rostedt
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 6 lines · Mathieu Desnoyers
* Steven Rostedt ([email protected]) wrote:
[...]Quoted 3 lines
Are these events now going to be labeled as stable? Is every tracepoint we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.
Quoted 7 lines
I'd like to bring up the point of KVM tracepoints here. KVM can be configured as a module, and may clearly contain tracepoints that we'd like to be stable.
My thought is that what we really want to enforce is "no stable tracepoints in drivers" rather than in "modules", but I might be wrong.
Thoughts ?
I still say no to stable tracepoints in modules. Once you open that door, everyone will have it.
But, that doesn't mean that a raw traepoint can't be stable. If the maintainer of that tracepoint states it is stable, then by all means, let tools use it.
-- Steve
Mathieu Desnoyers
reviewerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 13 lines · * Steven Rostedt ([email protected]) wrote:
Quoted 12 lines · Mathieu Desnoyers
* Steven Rostedt ([email protected]) wrote:
[...]Quoted 3 lines
Are these events now going to be labeled as stable? Is every tracepoint we have, much have the same data? Linus specifically said at Kernel Summit that he wants absolutely NO modules to have a stable tracepoint.
I'd like to bring up the point of KVM tracepoints here. KVM can be configured as a module, and may clearly contain tracepoints that we'd like to be stable.
My thought is that what we really want to enforce is "no stable tracepoints in drivers" rather than in "modules", but I might be wrong.
Quoted 8 lines
Quoted 1 line
Thoughts ?
I still say no to stable tracepoints in modules. Once you open that door, everyone will have it.
But, that doesn't mean that a raw traepoint can't be stable. If the maintainer of that tracepoint states it is stable, then by all means, let tools use it.
I'd really like to hear Avi's thoughts on this.
Thanks,
Mathieu
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com
Re: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 10 lines · Mathieu Desnoyers
Quoted 7 lines · * Steven Rostedt ([email protected]) wrote:
I still say no to stable tracepoints in modules. Once you open that door, everyone will have it.
But, that doesn't mean that a raw traepoint can't be stable. If the maintainer of that tracepoint states it is stable, then by all means, let tools use it.
I'd really like to hear Avi's thoughts on this.
Steven's scheme is fine with me. (kvm tracepoints are more or less stable, but all tools so far read the tracepoint definitions dynamically, and can cope with tracepoints being added/changed/removed).
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
In reply to Steven RostedtQuoted 3 lines · Steven Rostedt
I still say no to stable tracepoints in modules. Once you open that door, everyone will have it.
What about having KVM define the stable tracepoints in the built-in part of the kernel, if CONFIG_KVM is Y or M, and then export the tracepoints, such that the tracepoints can be called from a module?
That way the tracepoints aren't being *defined* in a module, they are just being *called* from the module. Does that seem like a reasonable compromise?
- Ted
Mathieu Desnoyers
reviewerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
Quoted 5 lines · * Ted Ts'o ([email protected]) wrote:
Quoted 3 lines · Steven Rostedt
I still say no to stable tracepoints in modules. Once you open that door, everyone will have it.
Quoted 7 lines
What about having KVM define the stable tracepoints in the built-in part of the kernel, if
CONFIG_KVMis Y or M, and then export the tracepoints, such that the tracepoints can be called from a module?That way the tracepoints aren't being *defined* in a module, they are just being *called* from the module. Does that seem like a reasonable compromise?
As Avi replied separately, he does not seem to think stable ABI is needed for KVM, given that his tools can deal with tracepoint addition/removal pretty well. However, let's keep this idea in mind if we ever face this issue with other "core" modules.
Thanks Ted,
Mathieu
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com
Steven Rostedt
maintainerRe: [RFC][PATCH 1/5] [PATCH 1/5] events: Add EVENT_FS the event filesystem
In reply to Greg KHQuoted 9 lines · Greg KH
Quoted 5 lines · Steven Rostedt
From: Steven Rostedt <[email protected]>
Copied mostly from debugfs, the eventfs is the filesystem that will include stable tracepoints. Currently nothing enables this filesystem as of this patch.
What? Wait, I wrote tracefs a long time ago just for this, why not take that code and use it instead?
Because:
1) I couldn't find it (I thought you called it eventfs, so I was searching for the wrong thing).
2) You never answered your ping on IRC
3) I was in a rush to get something out.
;-)
I'm perfectly fine in swapping this out. This is only an RFC anyway. This is also why I Cc'd you on this patch.
Quoted 3 lines
{sigh} And I just deleted that old tracefs git tree today as I thought that idea was long gone and dead....
{sigh} and it may still be dead :-(
-- Steve
Quoted 17 lines
Quoted 10 lines
Cc Greg KH <[email protected]>Signed-off-by Steven Rostedt <[email protected]>--- fs/Kconfig | 6 + fs/Makefile | 1 + fs/eventfs/Makefile | 4 + fs/eventfs/file.c | 53 ++++++ fs/eventfs/inode.c | 433 +++++++++++++++++++++++++++++++++++++++++++++++This seems a bit big, I don't think you need all of this for some reason. Are you sure you can't make it smaller?
thanks,
greg k-h
Steven Rostedt
maintainerRFC PATCH 2/5 tracing/events: Add code to (un)register stable events
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Add the framework to create and register stable events. To create a stable event, add a file into:
include/trace/stable/myfile.h
With the following format:
STABLE_EVENT(myevent,
EVENT_STRUCT( __field(type, item) __array(type, item, count) )
)
This will create the function prototype:
typedef void (*trace_proto_myevent)(type item, type *item);
And the functions:
int register_stable_trace_myevent(trace_proto_myevent func, void *data); void unregister_stable_trace_myevent(trace_proto_myevent func, void *data);
This will allow developers to register a callback of a stable event.
The stable events must have a matching raw (in field debugging) trace event. Although, only the names must match, the information in the stable event is just a subset of the information that can be extracted from the raw event.
--- include/trace/stable.h | 72 +++++++++++++++++++++++++++++++++++++++ include/trace/stable_list.h | 2 + kernel/Makefile | 1 + kernel/events/Makefile | 1 + kernel/events/event_reg.h | 79 +++++++++++++++++++++++++++++++++++++++++++ kernel/events/events.c | 16 +++++++++ kernel/trace/Kconfig | 1 + 7 files changed, 172 insertions(+), 0 deletions(-) create mode 100644 include/trace/stable.h create mode 100644 include/trace/stable_list.h create mode 100644 kernel/events/Makefile create mode 100644 kernel/events/event_reg.h create mode 100644 kernel/events/events.c
include/trace/stable.h new file +72 -0
@@ -0,0 +1,72 @@ +#ifndef _TRACE_STABLE_H +#define _TRACE_STABLE_H +/* +* stable.h - create structure and format for stable events +* +* Copyright (C) 2010 Steven Rostedt <[email protected]>, Red Hat Inc +* +* This file creates the registering function prototypes +* to hook to the stable events and the function prototypes of those +* hooks. +* +* The stable events themselves reside in the directory: +* include/trace/stable/ +* +* Each of these files must be added to the header: +* include/trace/stable_list. +* +* Code that registers and unregisters stable events only needs +* to include this file: +* #include <trace/stable.h> +* +* and all the stable structures and registering functions will +* also be included for all stable events. +* +* The current elements that can be used by stable events are: +* +* __field(type, item) +* __array(type, time, size) +* +* Example: +* STABLE_EVENT(myevent, +* EVENT_STRUCT( +* __field( pid_t, pid ) +* __array( char, comm, TASK_COMM_LEN ) +* ) +* ) +* +* The above would create the following: +* +* typedef void (*trace_proto_myevent)(pid_t pid, char comm[TASK_COMM_LEN]); +* int register_stable_trace_myevent(trace_proto_myevent func, void *data); +* void unregister_stable_trace_myevent(trace_proto_myevent func, void *data); +* +* These functions will be created in kernel/events/events.c +*/ + +#define STABLE_HEADER_MULTI_READ + +#undef __SEP__ +#define __SEP__ , + +#undef __field +#define __field (type, item) type item + +#undef __array +#define __array (type, item, size) type *item + +#undef EVENT_STRUCT +#define EVENT_STRUCT (s) s + +#undef STABLE_EVENT +#define STABLE_EVENT (name, estruct) \ + typedef void (*trace_proto_##name)(void *data, estruct); \ + int register_stable_trace_##name(trace_proto_##name func, void *data); \ + void unregister_stable_trace_##name(trace_proto_##name func, void *data) + +/* New stable defines must be added here */ +#include <trace/stable_list.h> + +#undef STABLE_HEADER_MULTI_READ + +#endif /* _TRACE_STABLE_H */
include/trace/stable_list.h new file +2 -0
@@ -0,0 +1,2 @@ + +/* New stable defines must be added here */
kernel/Makefile +1 -0
@@ -106,6 +106,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_event.o obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o obj-$(CONFIG_USER_RETURN_NOTIFIER) += user-return-notifier.o obj-$(CONFIG_PADATA) += padata.o +obj-y += events/ ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y) # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/events/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-y += events.o
kernel/events/event_reg.h new file +79 -0
79 lines not on this page · raw message
kernel/events/events.c new file +16 -0
@@ -0,0 +1,16 @@ +/* +* events.c - code to register stable events +* +* Copyright (C) 2010 Steven Rostedt <[email protected]> Red Hat Inc +* +* This file holds the hooks that are required to convert +* a raw tracepoing into a stable one. The conversion names +* must be named: hook_##tracepoint_name. +*/ +#include <linux/tracepoint.h> +#include <linux/module.h> +#include <trace/stable.h> + +static struct mutex stable_event_mutex; + +#include "event_reg.h"
kernel/trace/Kconfig +1 -0
@@ -94,6 +94,7 @@ config TRACING select NOP_TRACER select BINARY_PRINTF select EVENT_TRACING + select EVENT_FS config GENERIC_TRACER bool
Signature · 1 line
1.7.1
Steven Rostedt
maintainerRFC PATCH 3/5 tracing/events: Add infrastructure to show stable event formats
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Add the infrastructure to connect to the eventfs filesystem and create a event directory. Currently, all events are flat. That is there is no hierarchy. Each stable event has its own directory in /sys/kernel/events (after the eventfs is mounted there).
Currently only one file exists in each of the event directories. This file is the format file. The format shows each item in the stable event. The name of that item, the item's type, the size of the item (in bits), the count of entries of the item (if that item is an array), the alignment needed for that item to be read (arch specific) and the sign of the item.
--- kernel/events/Makefile | 2 +- kernel/events/event_format.c | 74 ++++++++++++++++++++++++++++++++++++++++++ kernel/events/event_format.h | 64 ++++++++++++++++++++++++++++++++++++ 3 files changed, 139 insertions(+), 1 deletions(-) create mode 100644 kernel/events/event_format.c create mode 100644 kernel/events/event_format.h
kernel/events/Makefile +1 -1
@@ -1 +1 @@ -obj-y += events.o +obj-y += events.o event_format.o
kernel/events/event_format.c new file +74 -0
74 lines not on this page · raw message
kernel/events/event_format.h new file +64 -0
64 lines not on this page · raw message
Signature · 1 line
1.7.1
Steven Rostedt
maintainerRFC PATCH 4/5 tracing/events: Add stable event sched_switch
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Add the stable event for sched_switch.
[root@bxf ~]# cat /sys/kernel/event/sched_switch/format
array:prev_comm type:char size:8 count:16 align:1 signed:1; field:prev_pid type:pid_t size:32 align:4 signed:1; field:prev_state type:char size:8 align:1 signed:1; array:next_comm type:char size:8 count:16 align:1 signed:1; field:next_pid type:pid_t size:32 align:4 signed:1;
--- include/trace/stable/sched.h | 20 ++++++++++++++++++++ include/trace/stable_list.h | 1 + kernel/events/events.c | 22 ++++++++++++++++++++++ 3 files changed, 43 insertions(+), 0 deletions(-) create mode 100644 include/trace/stable/sched.h
include/trace/stable/sched.h new file +20 -0
@@ -0,0 +1,20 @@ +#if !defined(_STABLE_SCHED_H) || defined(STABLE_HEADER_MULTI_READ ) +#define _STABLE_SCHED_H + +#include <linux/sched.h> + +/* +* Tracepoint for task switches, performed by the scheduler: +*/ +STABLE_EVENT (sched_switch, + +EVENT_STRUCT ( + __array( char, prev_comm, TASK_COMM_LEN )__SEP__ + __field( pid_t, prev_pid )__SEP__ + __field( char, prev_state )__SEP__ + __array( char, next_comm, TASK_COMM_LEN )__SEP__ +__field ( pid_t, next_pid ) + ) +); + +#endif /* _STABLE_SCHED_H */
include/trace/stable_list.h +1 -0
@@ -1,2 +1,3 @@/* New stable defines must be added here */ +#include <trace/stable/sched.h>
kernel/events/events.c +22 -0
@@ -13,4 +13,26 @@static struct mutex stable_event_mutex; +#include <trace/events/sched.h> + +DECLARE_TRACE (stable_sched_switch, + TP_PROTO(char *prev_comm, pid_t prev_pid, char prev_state, + char *next_comm, pid_t next_pid), +TP_ARGS (prev_comm, prev_pid, prev_state, next_comm, next_pid)); +DEFINE_TRACE (stable_sched_switch); + +static const char stat_nam[] =TASK_STATE_TO_CHAR_STR ; +static void hook_sched_switch (void *ignore, +struct task_struct *prev, +struct task_struct *next) +{ +unsigned state; + + state = prev->state ?__ffs (prev->state) +1 :0 ; + state = state <sizeof (stat_nam) -1 ? stat_nam[state] :'?' ; + +trace_stable_sched_switch (prev->comm, prev->pid, state, + next->comm, next->pid); +} +#include "event_reg.h"
Signature · 1 line
1.7.1
Steven Rostedt
maintainerRFC PATCH 5/5 tracing/events: Add sched_migrate_task stable event
In reply to Steven RostedtFrom: Steven Rostedt <[email protected]>
Add the stable event for sched_migrate_task.
[root@bxf ~]# cat /sys/kernel/event/sched_migrate_task/format
array:comm type:char size:8 count:16 align:1 signed:1; field:pid type:pid_t size:32 align:4 signed:1; field:orig_cpu type:int size:32 align:4 signed:1; field:dest_cpu type:int size:32 align:4 signed:1;
--- include/trace/stable/sched.h | 13 +++++++++++++ kernel/events/events.c | 10 ++++++++++ 2 files changed, 23 insertions(+), 0 deletions(-)
include/trace/stable/sched.h +13 -0
@@ -17,4 +17,17 @@ STABLE_EVENT(sched_switch, ) ); +/* +* Tracepoint for a task being migrated: +*/ +STABLE_EVENT (sched_migrate_task, + +EVENT_STRUCT ( + __array( char, comm, TASK_COMM_LEN )__SEP__ + __field( pid_t, pid )__SEP__ + __field( int, orig_cpu )__SEP__ +__field ( int, dest_cpu ) + ) +); +#endif /* _STABLE_SCHED_H */
kernel/events/events.c +10 -0
@@ -35,4 +35,14 @@ static void hook_sched_switch(void *ignore, next->comm, next->pid); } +DECLARE_TRACE (stable_sched_migrate_task, + TP_PROTO(char *comm, pid_t pid, int orig_cpu, int dest_cpu), +TP_ARGS (comm, pid, orig_cpu, dest_cpu)); +DEFINE_TRACE (stable_sched_migrate_task); + +static void hook_sched_migrate_task (void *ignore,struct task_struct *p,int dest_cpu) +{ +trace_stable_sched_migrate_task (p->comm, p->pid,task_cpu (p), dest_cpu); +} +#include "event_reg.h"
Signature · 1 line
1.7.1
Mathieu Desnoyers
reviewerQuoted 11 lines · * Steven Rostedt ([email protected]) wrote:
We also have name (redundant), ID (should be agnostic), and
print_fmt(lots of issues).So the new format looks like this:
[root@bxf ~]# cat
/sys/kernel/event/sched_switch/formatarray:prev_comm type:char size:8 count:16 align:1 signed:1; field:prev_pid type:pid_t size:32 align:4 signed:1; field:prev_state type:char size:8 align:1 signed:1; array:next_comm type:char size:8 count:16 align:1 signed:1; field:next_pid type:pid_t size:32 align:4 signed:1;
Hrm, this is mixing field and type definitions. How about we organize this in something that will be both parseable and extensible ?
First, I don't see what exporting the kernel-internal type "pid_t" in there gives you. Userspace knows nothing about this, so it seems pretty useless.
What do you think of this alternative layout ?
Named types below:
% cat /sys/kernel/event/types/char
parent = integer;
size = 8;
signed = true;
align = 8;
% cat /sys/kernel/event/types/pid_t
parent = integer;
size = 32;
signed = true;
align = 32; /* Or 8 if the architecture supports unaligned writes
efficiently */
% cat /sys/kernel/event/sched_switch/format
type { /* Nameless type */
parent = struct;
fields = {
{
type { /* Nameless type */
parent = array;
length = 16;
elem_type = char; /* refers to named type */
},
prev_comm,
},
{ pid_t, prev_pid, },
{ char, prev_state, },
{
type { /* Nameless type */
parent = array;
length = 16;
elem_type = char; /* refers to named type */
},
next_comm,
},
{ pid_t, next_pid, },
};
}With this layout, we can declare types like enumerations, e.g.
% cat /sys/kernel/event/types/trap_id_t
type {
parent = enum;
size = 5; /* 5-bit bitfield to hold the enumeration */
signed = false;
align = 1; /* bit-packed */
map = {
{ 0, "divide error" },
{ 2, "nmi stack" },
{ 4, "overflow" },
....
};
}So we can refer to this "named type" in all events for which we want to save trap ID ? We therefore get the mapping to a human-understandable name for free.
Quoted 3 lines
Some notes:
o The size is in bits.
Yep, this will immensely help when dealing with bitfields.
Quoted 2 lines
o We added an align, that is the natural alignment for the arch of that type.
Just watch out, in your initial example, I think your align field is in bytes rather than bits. Ideally we'd like everything to be consistent.
Thanks,
Mathieu
Quoted 4 lines
o We added an "array" type, that specifies the size of an element as well as a "count", where total size can be align(size) * count.
o We separated the field name from the type.
Signature · 4 lines
Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com