Skip to contentSkip to conversation

Using sched_clock for mmio-trace

DISCUSSION Jeff Muizelaar 6 participants 20 messages Feb 16, 2007 – Feb 18, 2007

lore
Share & tools
lore
#0 lore raw

I've built a tool with the goal of logging mmio writes and reads by device drivers. See http://nouveau.freedesktop.org/wiki/MmioTrace.

I'd like to add support for recording a time stamp on each read and write. Unfortunately, I am not sure which clock api I should use.

I had a look at blktrace and saw that it uses 'sched_clock()' for time stamps. However, this symbol is not exported to modules, and from what I've read it sounds like its use is discouraged.

The question is, what api should I be using? I need something that can be called from inside interrupt handlers, and obviously the more accurate and the lower the overhead the better.

-Jeff

#1 lore raw
Quoted 2 lines · Jeff Muizelaar

I've built a tool with the goal of logging mmio writes and reads by device drivers. See http://nouveau.freedesktop.org/wiki/MmioTrace.

FWIW, this is exactly a type of add-on trace patch that could be mooted by adoption of the ltt/systemtap "marker" facility. With it, you would not need so much code (e.g. no new user-space tools at all, reuse of common tracing buffer logic, permanently placed hooks) and would probably get more utility.

Quoted 3 lines

[...] The question is, what [timer] api should I be using? I need something that can be called from inside interrupt handlers, and obviously the more accurate and the lower the overhead the better.

We in systemtap land have the same problem, and so far made do with slightly postprocessed per-cpu TSC values.

- FChE

#2 lore raw
Quoted 11 lines · Frank Ch. Eigler
Quoted 2 lines · Jeff Muizelaar

I've built a tool with the goal of logging mmio writes and reads by device drivers. See http://nouveau.freedesktop.org/wiki/MmioTrace.

FWIW, this is exactly a type of add-on trace patch that could be mooted by adoption of the ltt/systemtap "marker" facility. With it, you would not need so much code (e.g. no new user-space tools at all, reuse of common tracing buffer logic, permanently placed hooks) and would probably get more utility.

Quoted 6 lines
Quoted 3 lines

[...] The question is, what [timer] api should I be using? I need something that can be called from inside interrupt handlers, and obviously the more accurate and the lower the overhead the better.

We in systemtap land have the same problem, and so far made do with slightly postprocessed per-cpu TSC values.

I've been working on a patch set (below), to expose the clocksources used by generic time to multiple users . It would allow timestamps from different clocks in a generic way. It's not merged, but I'd appreciate any input either of you might have..

ftp://source.mvista.com/pub/dwalker/clocksource/

Daniel

#3 lore raw
Quoted 6 lines · Daniel Walker

I've been working on a patch set (below), to expose the clocksources used by generic time to multiple users . It would allow timestamps from different clocks in a generic way. It's not merged, but I'd appreciate any input either of you might have..

ftp://source.mvista.com/pub/dwalker/clocksource/

Is it possible to see the resulting clocksource.h and maybe clocksource.c after the patch set? That would make looking at it much easier.

-Jeff

#10 lore raw
In reply to Frank Ch. Eigler
Quoted 10 lines · Frank Ch. Eigler
Quoted 2 lines · Jeff Muizelaar

I've built a tool with the goal of logging mmio writes and reads by device drivers. See http://nouveau.freedesktop.org/wiki/MmioTrace.

FWIW, this is exactly a type of add-on trace patch that could be mooted by adoption of the ltt/systemtap "marker" facility. With it, you would not need so much code (e.g. no new user-space tools at all, reuse of common tracing buffer logic, permanently placed hooks) and would probably get more utility.

Is there more information on this "marker" facility? e.g. what is a marker? Are they just like tracepoints?

Quoted 6 lines
Quoted 3 lines

[...] The question is, what [timer] api should I be using? I need something that can be called from inside interrupt handlers, and obviously the more accurate and the lower the overhead the better.

We in systemtap land have the same problem, and so far made do with slightly postprocessed per-cpu TSC values.

Thanks, I'll have a look at systemtap does. It's the stuff src/runtime/time.c right?

Also, you might want to move this stuff from the README to someplace more prominent on the systemtap website. It's nice not to have to dig far to get the source...

Download systemtap sources snapshot or from CVS:
ftp://sources.redhat.com/pub/systemtap/snapshots/ (or)
cvs -d :pserver:[email protected]:/cvs/systemtap login

# enter "anoncvs" as the password
cvs -d :pserver:[email protected]:/cvs/systemtap co src

-Jeff

#4 lore raw
In reply to Jeff Muizelaar
Quoted 11 lines · Jeff Muizelaar
Quoted 6 lines · Daniel Walker

I've been working on a patch set (below), to expose the clocksources used by generic time to multiple users . It would allow timestamps from different clocks in a generic way. It's not merged, but I'd appreciate any input either of you might have..

ftp://source.mvista.com/pub/dwalker/clocksource/

Is it possible to see the resulting clocksource.h and maybe clocksource.c after the patch set? That would make looking at it much easier.

Well you could just apply the patch set, but I stuck them in the same directory as above .. I'll delete them in 24 hours or so ..

At one point I replaced sched_clock() ,

ftp://source.mvista.com/pub/dwalker/clocksource/clocksource-v10/add_generic_sched_clock.patch

The API is similar to that version, and sched_clock was the simplest user of the API that I've done.

Daniel

#11 lore raw
In reply to Jeff Muizelaar
Quoted 6 lines · On Fri, 16 Feb 2007 13:30:14 -0500 Jeff Muizelaar wrote:
Quoted 5 lines · Frank Ch. Eigler
Quoted 2 lines · Jeff Muizelaar

I've built a tool with the goal of logging mmio writes and reads by device drivers. See http://nouveau.freedesktop.org/wiki/MmioTrace.

Quoted 8 lines
Quoted 5 lines

FWIW, this is exactly a type of add-on trace patch that could be mooted by adoption of the ltt/systemtap "marker" facility. With it, you would not need so much code (e.g. no new user-space tools at all, reuse of common tracing buffer logic, permanently placed hooks) and would probably get more utility.

Is there more information on this "marker" facility? e.g. what is a marker? Are they just like tracepoints?

On lkml for the past few days/months.
Look for "Linux Kernel Markers" in the subject line.

Mathieu, do you have a web site for LK Markers?

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

#18 lore raw
In reply to Jeff Muizelaar
Quoted 4 lines · Jeff Muizelaar

The question is, what api should I be using? I need something that can be called from inside interrupt handlers, and obviously the more accurate and the lower the overhead the better.

Use do_gettimeofday(). sched_clock() is not for general use and only for some very limited use cases and will give you unexpected results in several cases.

There are a few cases where gtod is still a little slow, but these are being addressed. In many cases it is fast. In some hardware it stays slow, but there is not much that can be done about that because of the hardware design.

It works fine from interrupt handlers and other strange contexts.

-Andi

#5 lore raw
In reply to Daniel Walker
Quoted 16 lines · Daniel Walker
Quoted 11 lines · Jeff Muizelaar
Quoted 6 lines · Daniel Walker

I've been working on a patch set (below), to expose the clocksources used by generic time to multiple users . It would allow timestamps from different clocks in a generic way. It's not merged, but I'd appreciate any input either of you might have..

ftp://source.mvista.com/pub/dwalker/clocksource/

Is it possible to see the resulting clocksource.h and maybe clocksource.c after the patch set? That would make looking at it much easier.

Well you could just apply the patch set, but I stuck them in the same directory as above .. I'll delete them in 24 hours or so ..

Quoted 6 lines

At one point I replaced sched_clock() ,

ftp://source.mvista.com/pub/dwalker/clocksource/clocksource-v10/add_generic_sched_clock.patch

The API is similar to that version, and sched_clock was the simplest user of the API that I've done.

Ok, so it would basically be:

init()

{
	clocksource *clock = clocksource_get_best_clock();
}

trace()

{
	trace.time = cyc2ns(clock, clocksource_read(clock));
}

This seems pretty sane to me.

The blk-trace code calibrates a per cpu offset for the sched_clock() time (see blk_trace_check_cpu_time and blk_trace_set_ht_offsets). Does the clocksource stuff help me with this or would I still need to do something like that?

I also noticed that you have a clocksource_get_masked_clock() call. This seems like a pretty awkward API to me. The first thing that came to mind when I read the name was 'what is a masked clock'. When I realized that it meant 'a clock w/o this flag', I still found it awkward that one has to specify what that don't want. e.g 'I don't want a clock that is not continuous.'

It think it would be better if you had sometime like 'clocksource_get_clock_with_features()' that took flags describing the needed characteristics instead of the unwanted ones.

e.g. clocksource_get_clock_with_features(CLOCKSOURCE_STABLE) or clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED)

instead of:

clocksource_get_clock_masked(CLOCKSOURCE_UNSTABLE) clocksource_get_clock_masked(CLOCKSOURCE_PM_AFFECTED)

Especially awkward is the CLOCKSOURCE_64BIT flag, as there isn't really anyway to specify that I want a 64bit timer, only a way to specify that I don't.

It also isn't clear what the implications of some of the flags are:
e.g:
NOT_CONTINUOUS - don't really have any idea what this means. UNSTABLE - this means the frequency can change right?
Does PM_AFFECTED imply UNSTABLE?
NOT_ATOMIC - does this affect me as user? PM_AFFECTED - it looks like the stp code deals with cpu speed
changing. Does the clocksource code do this for me with cyc2ns? If it does are there any reason I would want to avoid PM_AFFECTED clocks? If it doesn't how do I know that I need to correct it myself.

Hope this helps,

-Jeff

#19 lore raw
In reply to Andi Kleen
Quoted 10 lines · Andi Kleen
Quoted 4 lines · Jeff Muizelaar

The question is, what api should I be using? I need something that can be called from inside interrupt handlers, and obviously the more accurate and the lower the overhead the better.

Use do_gettimeofday(). sched_clock() is not for general use and only for some very limited use cases and will give you unexpected results in several cases.

Quoted 6 lines

There are a few cases where gtod is still a little slow, but these are being addressed. In many cases it is fast. In some hardware it stays slow, but there is not much that can be done about that because of the hardware design.

It works fine from interrupt handlers and other strange contexts.

Ok, I'll probably use it then. How does the overhead of calling do_gettimteofday() compare to doing an mmio read/write over PCI express? i.e. is it going to be a performance problem if I call do_gettimeofday for every mmio read/write?

Also, is there any good reason blk-trace doesn't use do_gettimeofday()?

-Jeff

#12 lore raw
In reply to Randy Dunlap
Quoted 14 lines · Randy Dunlap
Quoted 12 lines · On Fri, 16 Feb 2007 13:30:14 -0500 Jeff Muizelaar wrote:
Quoted 10 lines · Frank Ch. Eigler
Quoted 2 lines · Jeff Muizelaar

I've built a tool with the goal of logging mmio writes and reads by device drivers. See http://nouveau.freedesktop.org/wiki/MmioTrace.

FWIW, this is exactly a type of add-on trace patch that could be mooted by adoption of the ltt/systemtap "marker" facility. With it, you would not need so much code (e.g. no new user-space tools at all, reuse of common tracing buffer logic, permanently placed hooks) and would probably get more utility.

Quoted 8 lines
Quoted 2 lines

Is there more information on this "marker" facility? e.g. what is a marker? Are they just like tracepoints?

On lkml for the past few days/months.
Look for "Linux Kernel Markers" in the subject line.

Mathieu, do you have a web site for LK Markers?

Yeah, so if I understand correctly, markers are basically compile time locations that you can attach function calls to at run time, right?. If so, I don't think they are of much use to me.

What I am doing is page-faulting on every read or write to an mmio region, decoding the faulting instruction and passing the decoded information up to userspace through relayfs.

-Jeff

#6 lore raw
In reply to Jeff Muizelaar
Quoted 31 lines · Jeff Muizelaar
Quoted 16 lines · Daniel Walker
Quoted 11 lines · Jeff Muizelaar
Quoted 6 lines · Daniel Walker

I've been working on a patch set (below), to expose the clocksources used by generic time to multiple users . It would allow timestamps from different clocks in a generic way. It's not merged, but I'd appreciate any input either of you might have..

ftp://source.mvista.com/pub/dwalker/clocksource/

Is it possible to see the resulting clocksource.h and maybe clocksource.c after the patch set? That would make looking at it much easier.

Well you could just apply the patch set, but I stuck them in the same directory as above .. I'll delete them in 24 hours or so ..

Quoted 6 lines

At one point I replaced sched_clock() ,

ftp://source.mvista.com/pub/dwalker/clocksource/clocksource-v10/add_generic_sched_clock.patch

The API is similar to that version, and sched_clock was the simplest user of the API that I've done.

Ok, so it would basically be:

init()

{
	clocksource *clock = clocksource_get_best_clock();
}
Quoted 6 lines

trace()

{
	trace.time = cyc2ns(clock, clocksource_read(clock));
}

This seems pretty sane to me.

sched_clock has the down side of not taking into account rollover .. The scheduler doesn't care if it gets a few bad timestamps now and then, but that might screw up other code..

Also you could read cycles , and then later convert to nanoseconds when it's needed .. Which makes taking the timestamps a little faster.

Quoted 4 lines

The blk-trace code calibrates a per cpu offset for the sched_clock() time (see blk_trace_check_cpu_time and blk_trace_set_ht_offsets). Does the clocksource stuff help me with this or would I still need to do something like that?

Most of the clocksources are not per cpu .. But the tsc clocksource doesn't keep any per cpu offset information. There has been some work on syncing the TSC across cpus so I'm not sure to what extend an offset would be needed.

Quoted 6 lines

I also noticed that you have a clocksource_get_masked_clock() call. This seems like a pretty awkward API to me. The first thing that came to mind when I read the name was 'what is a masked clock'. When I realized that it meant 'a clock w/o this flag', I still found it awkward that one has to specify what that don't want. e.g 'I don't want a clock that is not continuous.'

yeah it is a little strange .

Quoted 9 lines

It think it would be better if you had sometime like 'clocksource_get_clock_with_features()' that took flags describing the needed characteristics instead of the unwanted ones.

e.g. clocksource_get_clock_with_features(CLOCKSOURCE_STABLE) or clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED)

One reason I did it the other way is because it's easier to specify what you know you don't want, than to specify all the things that you know you do want.

Almost everyone would want every flags listed,

clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED|CLOCKSOURCE_STABLE |CLOCKSOURC_ATOMIC|CLOCKSOURCE_64BITS|CLOCKSOURCE_CONTINUOUS);

Gets pretty ugly .. The clocksource interface already has a positive rating to describe the "best" clocks in the system, which is used to return the "best" clock .. Where the maintainers of the system give each clock a rating. I would imagine most people would just get the so called "best" clock which has the best rating..

I'm starting to think this long flags stringing effect could happen with negative flags also, but it's seems a lot less likely.

Quoted 8 lines

instead of

clocksource_get_clock_masked(CLOCKSOURCE_UNSTABLE) clocksource_get_clock_masked(CLOCKSOURCE_PM_AFFECTED)

Especially awkward is the CLOCKSOURCE_64BIT flag, as there isn't really anyway to specify that I want a 64bit timer, only a way to specify that I don't.

I might add a way to get specific flags, but I still think the flags should be mostly negative features.

Quoted 3 lines

It also isn't clear what the implications of some of the flags are:
e.g:
NOT_CONTINUOUS - don't really have any idea what this means.

It means the clock uses an interrupt to extend it's precision, or it's not a real cycle counter and depends only on an interrupt for timing.

Quoted 2 lines

UNSTABLE - this means the frequency can change right?
Does PM_AFFECTED imply UNSTABLE?

PM_AFFECTED means it could become unstable, and CLOCKSOURC_UNSTABLE means it's already become unstable.

Quoted 1 line

NOT_ATOMIC - does this affect me as user?

It could .. The PIT clock for example takes a spinlock. Some users might not want to use that clock cause they call from a context where that isn't acceptable (LTT for example). It's also slower to read from.

Quoted 5 lines

PM_AFFECTED - it looks like the stp code deals with cpu speed
changing. Does the clocksource code do this for me with cyc2ns? If it does are there any reason I would want to avoid PM_AFFECTED clocks? If it doesn't how do I know that I need to correct it myself.

There is a block notification system that lets you know when a clock becomes unstable . cyc2ns doesn't compensate for frequency changes . The TSC for example could fluctuate frequency pretty often. sched_clock for example uses the clock no matter what the state is , which is why I leave unstable clock in the list.

Quoted 1 line

Hope this helps,

Yeah, thanks for the feedback .

Daniel

#14 lore raw
In reply to Andi Kleen

Hi -

Quoted 4 lines · Andi Kleen
Quoted 2 lines

We in systemtap land have the same problem, and so far made do with slightly postprocessed per-cpu TSC values.

90+% likely you're not solving your problem correctly this way.

Yes, it was done as a last resort.

We need facility that we can call from even more demanding contexts than interrupt handlers, considering that kprobes can be placed nearly anywhere. This is one of the reasons why we don't just use good old do_gettimeofday(), since it takes locks and can lead to lock recursion if parts of itself are probed.

- FChE

#7 lore raw
In reply to Daniel Walker
Quoted 11 lines · Daniel Walker
Quoted 9 lines · Jeff Muizelaar

It think it would be better if you had sometime like 'clocksource_get_clock_with_features()' that took flags describing the needed characteristics instead of the unwanted ones.

e.g. clocksource_get_clock_with_features(CLOCKSOURCE_STABLE) or clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED)

Quoted 8 lines

One reason I did it the other way is because it's easier to specify what you know you don't want, than to specify all the things that you know you do want.

Almost everyone would want every flags listed,

clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED|CLOCKSOURCE_STABLE |CLOCKSOURC_ATOMIC|CLOCKSOURCE_64BITS|CLOCKSOURCE_CONTINUOUS);

I still meant for _with_features to have same semantics so calling:

clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED|CLOCKSOURCE_STABLE
|CLOCKSOURC_ATOMIC|CLOCKSOURCE_64BITS|CLOCKSOURCE_CONTINUOUS);

would be equivalent to calling:

clocksource_get_masked_clock(CLOCKSOURCE_PM_AFFECTED|CLOCKSOURCE_UNSTABLE
|CLOCKSOURC_NOT_ATOMIC|CLOCKSOURCE_64BITS|CLOCKSOURCE_NOT_CONTINUOUS);

The only difference is that the naming is reversed. i.e. it isn't a doulbe negative.

perhaps a better name would clocksource_get_clock_must_have() or clocksource_get_clock_must_be()

Quoted 8 lines

Gets pretty ugly .. The clocksource interface already has a positive rating to describe the "best" clocks in the system, which is used to return the "best" clock .. Where the maintainers of the system give each clock a rating. I would imagine most people would just get the so called "best" clock which has the best rating..

I'm starting to think this long flags stringing effect could happen with negative flags also, but it's seems a lot less likely.

The amount of flag stringing should be the same.

Quoted 6 lines
Quoted 6 lines

instead of

clocksource_get_clock_masked(CLOCKSOURCE_UNSTABLE) clocksource_get_clock_masked(CLOCKSOURCE_PM_AFFECTED)

Quoted 6 lines
Quoted 3 lines

Especially awkward is the CLOCKSOURCE_64BIT flag, as there isn't really anyway to specify that I want a 64bit timer, only a way to specify that I don't.

I might add a way to get specific flags, but I still think the flags should be mostly negative features.

Yeah, the problem is that all of the features are negative except for CLOCKSOURCE_64BIT, so you can't mask for it.

Quoted 25 lines
Quoted 3 lines

It also isn't clear what the implications of some of the flags are:
e.g:
NOT_CONTINUOUS - don't really have any idea what this means.

It means the clock uses an interrupt to extend it's precision, or it's not a real cycle counter and depends only on an interrupt for timing.

Quoted 2 lines

UNSTABLE - this means the frequency can change right?
Does PM_AFFECTED imply UNSTABLE?

PM_AFFECTED means it could become unstable, and CLOCKSOURC_UNSTABLE means it's already become unstable.

Quoted 1 line

NOT_ATOMIC - does this affect me as user?

It could .. The PIT clock for example takes a spinlock. Some users might not want to use that clock cause they call from a context where that isn't acceptable (LTT for example). It's also slower to read from.

Quoted 5 lines

PM_AFFECTED - it looks like the stp code deals with cpu speed
changing. Does the clocksource code do this for me with cyc2ns? If it does are there any reason I would want to avoid PM_AFFECTED clocks? If it doesn't how do I know that I need to correct it myself.

Quoted 5 lines

There is a block notification system that lets you know when a clock becomes unstable . cyc2ns doesn't compensate for frequency changes . The TSC for example could fluctuate frequency pretty often. sched_clock for example uses the clock no matter what the state is , which is why I leave unstable clock in the list.

It might be good if something like these explanations could be added as comments beside the feature flags.

So, if I was a systemtap style user of the clocksource api, I'd still have to do something like:

init() {
	// assume this gives me the tsc
	clock = clocksource_get_masked_clock(CLOCKSOURCE_NOT_ATOMIC);
	register a cpu_freq notifier
}

trace() {
trace.time = compute_cyc2_ns_by_hand_using_info_from_cpu_freq(clocksource_read(clock))
}

In this case it doesn't look like using the clocksource stuff helps this style of user much as all that is abstracted away is really the reading the tsc. On the other hand it sounds like if you want all this stuff taken care of for you, most people should be using do_gettimeofday() instead.

-Jeff

#8 lore raw
Quoted 6 lines · Jeff Muizelaar

I still meant for _with_features to have same semantics so calling:

clocksource_get_clock_with_features(CLOCKSOURCE_PM_UNAFFECTED|CLOCKSOURCE_STABLE
|CLOCKSOURC_ATOMIC|CLOCKSOURCE_64BITS|CLOCKSOURCE_CONTINUOUS);

Quoted 7 lines

would be equivalent to calling:

clocksource_get_masked_clock(CLOCKSOURCE_PM_AFFECTED|CLOCKSOURCE_UNSTABLE
|CLOCKSOURC_NOT_ATOMIC|CLOCKSOURCE_64BITS|CLOCKSOURCE_NOT_CONTINUOUS);

The only difference is that the naming is reversed. i.e. it isn't a doulbe negative.

Yes, I assumed that ..

Quoted 9 lines

perhaps a better name would clocksource_get_clock_must_have() or clocksource_get_clock_must_be()

Quoted 6 lines

Gets pretty ugly .. The clocksource interface already has a positive rating to describe the "best" clocks in the system, which is used to return the "best" clock .. Where the maintainers of the system give each clock a rating. I would imagine most people would just get the so called "best" clock which has the best rating..

Quoted 4 lines
Quoted 2 lines

I'm starting to think this long flags stringing effect could happen with negative flags also, but it's seems a lot less likely.

The amount of flag stringing should be the same.

I don't think so .. The common case with negative flags is no flags, then next would be CLOCKSOURCE_UNSTABLE. At most I would guess two flags .. The other direction your likely to have people using all flags most of the time. That's why I showed a function call with all the flags listed.

To be clear the clocksources are already sorted with a rating (no flags involved).

So for example, On i386 the clocks are sorts as follows,

tsc (300)
acpi_pm (200)
pit (110)
jiffies (1)

So the bulk of the positive information is already there for you. The tsc clock is by far the best in the system if it's stable. If you ignore the flags, then you will always be given the tsc .

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

instead of

clocksource_get_clock_masked(CLOCKSOURCE_UNSTABLE) clocksource_get_clock_masked(CLOCKSOURCE_PM_AFFECTED)

Especially awkward is the CLOCKSOURCE_64BIT flag, as there isn't really anyway to specify that I want a 64bit timer, only a way to specify that I don't.

Quoted 5 lines
Quoted 2 lines

I might add a way to get specific flags, but I still think the flags should be mostly negative features.

Yeah, the problem is that all of the features are negative except for CLOCKSOURCE_64BIT, so you can't mask for it.

It's meant as a negative feature. So you can mask it if you can't handle the math .. The only 64bit clock I know off is the tsc, and it's got the highest rating of all clocks.

Quoted 2 lines

It might be good if something like these explanations could be added as comments beside the feature flags.

Ok .. On my TODO list ..

Quoted 14 lines

So, if I was a systemtap style user of the clocksource api, I'd still have to do something like:

init() {
	// assume this gives me the tsc
	clock = clocksource_get_masked_clock(CLOCKSOURCE_NOT_ATOMIC);
	register a cpu_freq notifier
}

trace() {
trace.time = compute_cyc2_ns_by_hand_using_info_from_cpu_freq(clocksource_read(clock))
}

Quoted 5 lines

In this case it doesn't look like using the clocksource stuff helps this style of user much as all that is abstracted away is really the reading the tsc. On the other hand it sounds like if you want all this stuff taken care of for you, most people should be using do_gettimeofday() instead.

Instead of using an unstable clock and trying to compensate for it, the solution is not to use unstable clocks. The clocksource stuff does help with that.. You can either use the tsc until it becomes unstable, or use the acpi_pm clock which never becomes unstable. Both are used in the same way.

Quoted 1 line

init() {

	  // gives you the acpi_pm
	  clock = clocksource_get_masked_clock(CLOCKSOURCE_PM_AFFECTED|CLOCKSOURCE_NOT_ATOMIC);
Quoted 5 lines

register a cpu_freq notifier
}

trace() {
trace.time =

cyc2ns(clock, clocksource_read(clock))

Quoted 2 lines

}

Daniel

#9 lore raw
Quoted 9 lines · Daniel Walker
Quoted 9 lines
Quoted 8 lines

Gets pretty ugly .. The clocksource interface already has a positive rating to describe the "best" clocks in the system, which is used to return the "best" clock .. Where the maintainers of the system give each clock a rating. I would imagine most people would just get the so called "best" clock which has the best rating..

I'm starting to think this long flags stringing effect could happen with negative flags also, but it's seems a lot less likely.

Quoted 7 lines
Quoted 1 line

The amount of flag stringing should be the same.

I don't think so .. The common case with negative flags is no flags, then next would be CLOCKSOURCE_UNSTABLE. At most I would guess two flags .. The other direction your likely to have people using all flags most of the time. That's why I showed a function call with all the flags listed.

I think you still misunderstand me. The common case is still no flags.

clocksource_get_clock_must_have(0) would return clocks that are stable and unstable. clocksource_get_clock_must_have(CLOCKSOURCE_STABLE) would only return clocks that are stable, just like clocksource_get_clock_masked(CLOCKSOURCE_UNSTABLE) only returns clocks that are stable.

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 8 lines

instead of

clocksource_get_clock_masked(CLOCKSOURCE_UNSTABLE) clocksource_get_clock_masked(CLOCKSOURCE_PM_AFFECTED)

Especially awkward is the CLOCKSOURCE_64BIT flag, as there isn't really anyway to specify that I want a 64bit timer, only a way to specify that I don't.

I might add a way to get specific flags, but I still think the flags should be mostly negative features.

Quoted 6 lines
Quoted 2 lines

Yeah, the problem is that all of the features are negative except for CLOCKSOURCE_64BIT, so you can't mask for it.

It's meant as a negative feature. So you can mask it if you can't handle the math .. The only 64bit clock I know off is the tsc, and it's got the highest rating of all clocks.

Ah ok, I see that now. Maybe CLOCKSOURCE_OVER_32BITS would be a better name? It might convey the negativity better...

-Jeff

#15 lore raw
In reply to Frank Ch. Eigler
Quoted 3 lines

This is one of the reasons why we don't just use good old do_gettimeofday(), since it takes locks and can lead to lock recursion if parts of itself are probed.

do_gettimeofday doesn't take locks.

Only restriction is that you can't single step it with long pauses between instructions.

-Andi

#16 lore raw
Quoted 8 lines · Andi Kleen
Quoted 3 lines

This is one of the reasons why we don't just use good old do_gettimeofday(), since it takes locks and can lead to lock recursion if parts of itself are probed.

do_gettimeofday doesn't take locks.

Only restriction is that you can't single step it with long pauses between instructions.

Err, it uses read side of xtime lock, so you can not call it from a place which write locks xtime lock.

tglx

#17 lore raw
Quoted 5 lines · Thomas Gleixner
Quoted 4 lines · Andi Kleen
Quoted 3 lines

This is one of the reasons why we don't just use good old do_gettimeofday(), since it takes locks and can lead to lock recursion if parts of itself are probed.

Quoted 7 lines
Quoted 4 lines

do_gettimeofday doesn't take locks.

Only restriction is that you can't single step it with long pauses between instructions.

Err, it uses read side of xtime lock, so you can not call it from a place which write locks xtime lock.

Err, you can -- seqlocks never deadlock.

The only thing that doesn't work is to single step with long enough pauses with interrupts on inbetween that the sequence numbers increase:
you get a livelock then.

-Andi