I request inclusion of SAS Transport Layer and AIC-94xx into the kernel
Subsystems (4)
cover ktimer reworked
this patch series is a refactored version of the ktimer-subsystem patch.
firstly, we fixed two bugs noticed by Roman Zippel (and mentioned in the ptimer patch-queue):
- clock_nanosleep(): interrupted absolute sleeps are now handled correctly
- overrun accounting in posix interval timers is fixed.
furthermore, we have split the ktimer patch series up into 23 patches - each stage introduces one new (and mostly small) conceptual step. Each stage builds & boots fine. The ktimer design has not been changed radically, but some improvements and reshaping has been done in the course of the rework.
based on the idea from Roman Zippel, we have eliminated the 'pure scalar type' in ktime.h, and have gone for the ktime union for both 32-bit and 64-bit platforms. This resulted in further simplifications, e.g. the ktime_cmp() ugliness has been removed.
to address the naming debate: we do agree that 'struct ktimer' and 'struct timer_list' is confusing and awkward - both are about timers. Same goes for kernel/ktimer.c and kernel/timer.c.
but what we'd like to achieve as an end-result is the clear separation of 'timer' vs. 'timeout' APIs. Our proposed end result would be to have 'struct ktimer' for timers, and 'struct ktimeout' for timeouts.
to prove the feasibility of this separation, we have also included another 20 patches ontop of the ktimer patch-queue, which patches introduce the 'ktimeout' types and APIs. This is mostly a careful renaming and refactoring of the current 'struct timer_list' and APIs into a 'struct ktimeout' object and API. (see the patches for details) These patches too build & boot at every stage.
we have done this to show that the 'ktimer' vs. 'timer_list' confusion is only temporary, and that the intended end-result is 'struct ktimer' and 'struct ktimeout' - two separate concepts.
on the other hand, 'struct ptimer' and 'struct ktimeout' [or 'struct ptimer' and 'struct timer_list'] would be just as confusing.
here's a list of other differences to the ptimer patchqueue:
- ptimer.c is smaller, for the price of making users of the APIs more complex. ktimer.c carries more features and thus more code, so that users can have simpler code.
these features are not unconditional: we tried to be careful to not let 'cruft' creep into ktimer.c: e.g. quite some Posix 'special timer code' is still kept in posix-timers.c.
Right now the extra code in ktimer.c is roughly offset by the code reduction in posix-timers.c, itimer.c and timer.c/ktimeout.c, and we have not converted posix-cpu-timers to ktimers yet. So we believe this choice of us is a net win, even with the relative low number of ktimer users at this early stage.
- the ktimer subsystem is designed with the extensibility for high resolution timers and dynamic ticks in mind, without having to do further rewrites. The practical feasibility and cleanliness of the ktimer approach has been proven since the very beginning, both the -khrt and the -rt trees have carried those patches for months now, with a real HRT implementation ontop of it. We believe that some details that the ptimers patch-queue chopped off the ktimers codebase will have to be reintroduced for HRT timers later on.
- the resolution handling is implemented without any jiffy relations and resembles the behaviour of the current implementation. The first view of more complexity has to be carefully weighed against the flexibility for further extensions.
- ktimer is fully docbook documented and well commented. The ptimer patchqueue was based off an older ktimer tree with less comments and no documentation.
if there are any other substantial differences between the ptimer patchqueue and the current ktimer queue then please speak up. (there was no documentation of all ktimer->ptimer changes, so we might have missed something)
the patchset has been compiled & booted against recent -linus trees, on x86 and x64.
The patch series is also available from
http://www.tglx.de/projects/ktimers/patch-2.6.15-rc2-kt-rework.tar.bz2
Thomas, Ingo
--
Re: [patch 00/43] ktimer reworked
Quoted 2 lines · Thomas Gleixner
this patch series is a refactored version of the ktimer-subsystem patch.
25 files changed, 3364 insertions(+), 1827 deletions(-)
allnoconfig, before:
text data bss dec hex filename
764888 157221 53748 975857 ee3f1 vmlinux
after:
text data bss dec hex filename
766712 157741 53748 978201 eed19 vmlinux
Remind me what we gained for this?
Re: [patch 00/43] ktimer reworked
Quoted 11 lines · Andrew Morton
Quoted 2 lines · Thomas Gleixner
this patch series is a refactored version of the ktimer-subsystem patch.
25 files changed, 3364 insertions(+), 1827 deletions(-)allnoconfig, before:
text data bss dec hex filename
764888 157221 53748 975857 ee3f1 vmlinux
Quoted 6 lines
after:
text data bss dec hex filename
766712 157741 53748 978201 eed19 vmlinuxRemind me what we gained for this?
well, for 1824 bytes of code [*] and 520 bytes of data you got a new, clean timer subsystem, which is per-clock tree based and hres-timers ready. It also doesnt scan all active timers linearly and fixes them up whenever NTP decides to mend the clock a bit. It also has no jiffy dependencies and has nsec resolution with timeouts of up to 292 years, to the nanosec. It has no subjiffies, no HZ, no tradeoffs.
note that ktimer.o itself is larger than 1824 bytes:
size kernel/ktimer.o
text data bss dec hex filename
3912 100 0 4012 fac kernel/ktimer.o
so it has already offset roughly half of its size.
we can (and will) try to improve it further, but if anyone desires to get it for free, that's probably not possible. (only 'probable' because we have not converted posix-cpu-timers yet, another ktimer conversion candidate with code reduction potential)
it had to be a new set of APIs, which all take text space. We'll try to shave off some more .text, but miracles are not expected.
Ingo
[*] if you enable CONFIG_KTIME_SCALAR, then on x86 we get denser
ktime_t code. We keep it off by default to give the union representation testing (the scalar representation is the more trivial case). It should shave off another 300 bytes from your kernel's size. We'll probably enable KTIME_SCALAR on x86 later on.
Re: [patch 00/43] ktimer reworked
In reply to Thomas GleixnerHi,
Quoted 7 lines · Thomas Gleixner
to address the naming debate: we do agree that 'struct ktimer' and 'struct
timer_list' is confusing and awkward - both are about timers. Same goes forkernel/ktimer.candkernel/timer.c.but what we'd like to achieve as an end-result is the clear separation of 'timer' vs. 'timeout' APIs. Our proposed end result would be to have 'struct ktimer' for timers, and 'struct ktimeout' for timeouts.
Sorry, but calling it "ktimeout" would be completely wrong.
"timeout" is a rather imprecise term, which can have different meanings depending on the context, e.g. any timer usually has a "timeout value", but what is meant here is a "timeout timer". So basically this is supposed to be about "timer" vs "timeout timer".
First, the problem here is these "timeout timer" are not restricted to just a single use case, so there is no reason to name them like this only because they are most commonly used as timeout timer in the kernel. There are many types of timer usages in the kernel and there is no reason to give every one of them their own API.
Second, "timer" is a generic term, which would include all types of timer and this suggests that this is a generally usable timer. The emphasis of this timer is to provide precise, high resolution timer, but this purpose is not reflected in the name at all.
This is a really bad choice in names, both are timer. timer_list is so far _the_ generic API for _any_ type of timer, this unlikely to change for some time, so restricting this via a new name is confusing and wrong. The focus of the new timer system is high resolution timer and a good name would include something which describes this purpose and would clearly distinguish it from the current common timer.
Calling them "timer" and "timeout" would completely reverse the rolls, which makes it completely wrong.
Quoted 3 lines
-
ptimer.cis smaller, for the price of making users of the APIs more complex.ktimer.ccarries more features and thus more code, so that users can have simpler code.
What complexity are you talking about? Let's look at the itimer:
int it_real_fn(struct ptimer *timer)
{
struct signal_struct *sig = container_of(timer, struct signal_struct, real_timer);
send_group_sig_info(SIGALRM, SEND_SIG_PRIV, sig->tsk);
if (sig->it_real_incr.tv64 == KTIME_ZERO)
return 0;
sig->real_timer.expires = ktime_add(timer->base->last_expired, sig->it_real_incr);
return 1;
}This looks really simple to me. The last few lines could be moved into the timer code, but I decided against it because this version leaves more flexibility to the user. Currently we have only few users, should we have more users we could for example change it to:
return ptimer_rearm(timer, sig->it_real_incr);
But this is currently not needed, we can still refine the API as soon as there are more users, so we see what is really needed. What my patch does is to provide the basic functionality upon which further improvements are possible.
Posix timer are OTOH indeed more complex, but here I left the complexity there instead of moving it into ptimer.
Quoted 8 lines
- the ktimer subsystem is designed with the extensibility for high resolution timers and dynamic ticks in mind, without having to do further rewrites. The practical feasibility and cleanliness of the ktimer approach has been proven since the very beginning, both the -khrt and the -rt trees have carried those patches for months now, with a real HRT implementation ontop of it. We believe that some details that the ptimers patch-queue chopped off the ktimers codebase will have to be reintroduced for HRT timers later on.
This is very vague. What kind of "further rewrites" will be required. What are the "details" you "believe" are so important.
Quoted 4 lines
- the resolution handling is implemented without any jiffy relations and resembles the behaviour of the current implementation. The first view of more complexity has to be carefully weighed against the flexibility for further extensions.
Again, what "further extensions"?
ptimer is not limited to jiffy resolutions, it's the resolution it _currently_ uses and it's the best currently possible with the current clock abstraction.
The resolution handling in your patch is overly complex, most of the rounding is off and sometimes even wrong. I explained the details of the rounding in the ktime_t patch and I would like to encourage you to pick up from there and explain what the hell you're talking about.
Quoted 3 lines
- ktimer is fully docbook documented and well commented. The ptimer patchqueue was based off an older ktimer tree with less comments and no documentation.
I intentionally left out some of the documentation, because I wanted some discussion about the implementation first and then update the documentation based on the discussion, so it reflects a common view instead of finishing the discussion before it even started.
Quoted 4 lines
if there are any other substantial differences between the ptimer patchqueue and the current ktimer queue then please speak up. (there was no documentation of all
ktimer->ptimerchanges, so we might have missed something)
If there is anything unclear, you could just ask.
Quoted 2 lines
The patch series is also available from
http://www.tglx.de/projects/ktimers/patch-2.6.15-rc2-kt-rework.tar.bz2
I don't want to go into the details here. Most of the initial cleanup patches could easily be done afterwards and are not critical enough to be done first, the ktimer part needs an explanation what the extra complexity is needed for and I explained above what I think about the last part.
BTW a quick test shows the overrun handling is still broken. timer_gettime seems to be broken now for these cases as well.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 7 lines · Roman Zippel
Quoted 1 line · Thomas Gleixner
but what we'd like to achieve as an end-result is the clear separation of 'timer' vs. 'timeout' APIs. Our proposed end result would be to have 'struct ktimer' for timers, and 'struct ktimeout' for timeouts.
Sorry, but calling it "ktimeout" would be completely wrong.
"timeout" is a rather imprecise term, which can have different meanings depending on the context, e.g. any timer usually has a "timeout value", but what is meant here is a "timeout timer". So basically this is supposed to be about "timer" vs "timeout timer". [snip lengthy discussion]
If I recall correctly, this whole naming mess has been discussed to death before, with the result that almost everybody but Roman thought the names were perfectly clear such that a timer is _expected_ to expire and a timeout is not, therefore timers should be optimized for add=>run=>expire and timeouts optimized for add=>run=>remove.
Cheers,
Kyle Moffett
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/E/U d- s++: a18 C++++>$ ULBX*++++(+++)>$ P++++(+++)>$ L++++(+++)>$ !E- W+++(++) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+ t+(+++) 5 X R? !tv-(--) b++++(++) DI+(++) D+++ G e>++++$ h*(+)>++$ r%(--) !y?-(--) ------END GEEK CODE BLOCK------
Re: [patch 00/43] ktimer reworked
Hi,
Quoted 5 lines · Kyle Moffett
If I recall correctly, this whole naming mess has been discussed to death before, with the result that almost everybody but Roman thought the names were perfectly clear such that a timer is _expected_ to expire and a timeout is not, therefore timers should be optimized for add=>run=>expire and timeouts optimized for add=>run=>remove.
The human language is a bit more complicated than this (at least English and related languages). Depending on the context a word can have different meanings, e.g. if you ask an athlete what "timeout" means, you'll get a different answer than you would get from an engineer. Even if we limit it to the technical field one can define "timeout" very generally as "a period of time after which an event is generated". Does this imply this timeout is usually aborted? For some people it obviously does, but I highly doubt this is generally true. Without any context "timeout" can mean many similiar, but still different things. If you don't provide any context, it will trigger different associations and people will add their own context of how they use "timeout". You will of course find a large overlap, but the less context you provide, the more likely are misunderstandings.
A good name provides enough context to minimize misunderstandings, the name is important for how people will perceive and use something. Here we get to a larger problem, which goes beyond simple naming issues. Thomas and Ingo seem to want to completely redefine how time is managed in the kernel. The consequences for this would be very farreaching and should be discussed independently. Discussing this under the topic of high resolution timer would provide the entirely wrong context and lead to misunderstandings.
Whatever it is Thomas and Ingo are trying to do with the current kernel timer, they have to explain it in the proper context. I'm not going to second-guess their intentions and sneaking these changes in as part of the high resolution timer is unacceptable.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 4 lines · Roman Zippel
The human language is a bit more complicated than this (at least English and related languages). Depending on the context a word can have different meanings, e.g. if you ask an athlete what "timeout" means, you'll get a different answer than you would get from an engineer.
Actually, no you won't. The athlete will say "A timeout? Something out of the ordinary happened, and coach wants me to go to the sidelines to talk." Timeouts are unexpected and exceptional, whether you're an athlete or a piece of code. On the other hand, they have a timer that everyone *expects* to expire at the end of the quarter or game.
Ray, who is both an athlete and a native English speaker, who thinks the naming is the clearest of anything to come across this list in ages.
Re: [patch 00/43] ktimer reworked
Quoted 12 lines · Ray Lee
Quoted 4 lines · Roman Zippel
The human language is a bit more complicated than this (at least English and related languages). Depending on the context a word can have different meanings, e.g. if you ask an athlete what "timeout" means, you'll get a different answer than you would get from an engineer.
Actually, no you won't. The athlete will say "A timeout? Something out of the ordinary happened, and coach wants me to go to the sidelines to talk." Timeouts are unexpected and exceptional, whether you're an athlete or a piece of code. On the other hand, they have a timer that everyone *expects* to expire at the end of the quarter or game.
Quoted 3 lines
Ray, who is both an athlete and a native English speaker, who thinks the naming is the clearest of anything to come across this list in ages.
rmk, also a native English speaker, agrees with Ray, Thomas and Ingo. As does dictionary.reference.com's definitions of timeout and timer:
timeout
A period of time after which an error condition is raised if some event has not occured. A common example is sending a message. If the receiver does not acknowledge the message within some preset timeout period, a transmission error is assumed to have occured.
timer
a timepiece that measures a time interval and signals its end
Hence, timers have the implication that they are _expected_ to expire. Timeouts have the implication that their expiry is an exceptional condition.
So can we stop rehashing this stupid discussion?
Signature · 3 lines
Russell King Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/ maintainer of: 2.6 Serial core
Re: [patch 00/43] ktimer reworked
Hi,
Quoted 7 lines · Russell King
timeout
A period of time after which an error condition is raised if some event has not occured. A common example is sending a message. If the receiver does not acknowledge the message within some preset timeout period, a transmission error is assumed to have occured.
Quoted 7 lines
timer
a timepiece that measures a time interval and signals its end
Hence, timers have the implication that they are _expected_ to expire. Timeouts have the implication that their expiry is an exceptional condition.
IOW a timeout uses a timer to implement an exceptional condition after a period of time expires.
Quoted 1 line
So can we stop rehashing this stupid discussion?
The naming isn't actually my primary concern. I want a precise definition of the expected behaviour and usage of the old and new timer system. If I had this, it would be far easier to choose a proper name. E.g. I still don't know why ktimeout should be restricted to raise just "error conditions", as the name implies.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 3 lines · Roman Zippel
Hi,
On Thu, 1 Dec 2005, Russell King wrote:
...
Quoted 7 lines
Quoted 3 lines
Hence, timers have the implication that they are _expected_ to expire. Timeouts have the implication that their expiry is an exceptional condition.
IOW a timeout uses a timer to implement an exceptional condition after a period of time expires.
Quoted 8 lines
Quoted 1 line
So can we stop rehashing this stupid discussion?
The naming isn't actually my primary concern. I want a precise definition of the expected behaviour and usage of the old and new timer system. If I had this, it would be far easier to choose a proper name. E.g. I still don't know why ktimeout should be restricted to raise just "error conditions", as the name implies.
ktimeout may not need to be restricted to anything.
It should just be documented simply as: If you need to set some timed event to happen that you don't expect to occur then use a ktimeout. Where this timed event is an event that lets you know that another event hasn't happened in a given time. (I want to know if x didn't happen by this time).
If a timed event is expected to occur then use ktimer. Where it is mostly used for the event itself (I want x to happen at this time).
Now you could use ktimout on something that will always occur, but this will just be inefficient, since ktimeout is optimized for removal and not expiration. Which usually happens when something "times out".
And you could use ktimer on something that isn't going to occur. But again this is just inefficient.
So Roman, please have someone else speak up and let us know that they are just as confused on these names as you are. Currently, it seems that you are the only one that doesn't understand the difference between a timeout and a timer. You seem very intelligent and that could be why you are getting confused. You're looking too deep into the implementation of timers and timeouts, where they seem to use each other. You just need to take a step back and look at this at a higher view. Think about what to use when you need to implement being told when something has timed out (timeout) or when you just want to do something that happens a a certain time (timer).
-- Steve
Re: [patch 00/43] ktimer reworked
Hi,
Quoted 8 lines · Steven Rostedt
It should just be documented simply as: If you need to set some timed event to happen that you don't expect to occur then use a ktimeout. Where this timed event is an event that lets you know that another event hasn't happened in a given time. (I want to know if x didn't happen by this time).
If a timed event is expected to occur then use ktimer. Where it is mostly used for the event itself (I want x to happen at this time).
If that is really _the_ defining difference, then we are _seriously_ screwed.
Here are a few items I would consider in choosing a timer:
- reading time: to program a timer you have to read the time first, reading jiffies is practically free, whereas reading the precise time can be very expensive. With the right hardware it can be optimized to be quite cheap, but if portability is important you may want to avoid the extra cost.
- calculations: jiffies is a long integer whereas ktime_t is 64bit, so if you need a lot of complex time calculation, you should take the cost for 32bit archs into account.
- resolution: how precise must the timer be? jiffies can't represent time values less than 1ms, but if time is e.g. measured in 10th of a second, jiffies may be enough.
- timer life time: if only a short interval is needed (e.g. a fraction of a second) timer_list is often a lot cheaper.
Quoted 2 lines
So Roman, please have someone else speak up and let us know that they are just as confused on these names as you are.
Let's ignore the name for a moment, let's instead prioritize the above list.
If your item of whether a timer does expire or not is really the most important criteria for choosing a timer, I will accept any name you want.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 1 line · Roman Zippel
Hi,
Thank you Roman, this has cleared up a little where you are coming from. The name issue should never have been brought up, and what you state here should have been mentioned first. (If you did mention this earlier, I guess it was just lost in the noise).
Quoted 9 lines · Steven Rostedt
Quoted 7 lines
It should just be documented simply as: If you need to set some timed event to happen that you don't expect to occur then use a ktimeout. Where this timed event is an event that lets you know that another event hasn't happened in a given time. (I want to know if x didn't happen by this time).
Quoted 5 lines
Quoted 2 lines
If a timed event is expected to occur then use ktimer. Where it is mostly used for the event itself (I want x to happen at this time).
If that is really _the_ defining difference, then we are _seriously_ screwed.
I'm not convinced that we are.
Quoted 8 lines
Here are a few items I would consider in choosing a timer:
- reading time: to program a timer you have to read the time first, reading jiffies is practically free, whereas reading the precise time can be very expensive. With the right hardware it can be optimized to be quite cheap, but if portability is important you may want to avoid the extra cost.
If you are only concerned with ms resolution, then the added expense of precise time is pretty negligible. And these times are still for usage with timers that are expected to expire. Remember that timeouts are still using jiffies.
As for portablility, I believe John Stultz has some nice plugins coming to what timer source you want to use, so if there's a better way to get a time, these should make things easy to add.
Quoted 4 lines
- calculations: jiffies is a long integer whereas
ktime_tis 64bit, so if you need a lot of complex time calculation, you should take the cost for 32bit archs into account.
Again, the cost is negligible for ms resolution. And we're getting close to 2038 so we need to be thinking in 64 bits anyway ;)
I'm not running huge multiuser systems, so I don't know what the average amount of timers that are used (ones that are expected to expire), and how much of an overhead it would be for these timers to be using 64 bit calculations and precise timers.
Quoted 4 lines
- resolution: how precise must the timer be? jiffies can't represent time values less than 1ms, but if time is e.g. measured in 10th of a second, jiffies may be enough.
And they would be if that is all you need. But coming from an embedded point of view, that is not nearly enough. I really see HighRes making it into the kernel soon, and any new code in this area really needs to take that into account.
Quoted 3 lines
- timer life time: if only a short interval is needed (e.g. a fraction of a second)
timer_listis often a lot cheaper.
And again, you are only limited to 1000 choices to go off in that fraction of a second if jiffies is the resolution (with jiffies an 1000HZ).
As for the timer_list, you mean a linear list? If you have a bunch of timers to go off within a fraction of a second, couldn't a linear list actually take longer?
-- Steve
Re: [patch 00/43] ktimer reworked
Hi,
Quoted 1 line · Steven Rostedt
The name issue should never have been brought up,
Well, I haven't brought it up, I'm trying to get rid od it.
Quoted 6 lines
Quoted 5 lines
- reading time: to program a timer you have to read the time first, reading jiffies is practically free, whereas reading the precise time can be very expensive. With the right hardware it can be optimized to be quite cheap, but if portability is important you may want to avoid the extra cost.
Quoted 8 lines
If you are only concerned with ms resolution, then the added expense of precise time is pretty negligible. And these times are still for usage with timers that are expected to expire. Remember that timeouts are still using jiffies.
As for portablility, I believe John Stultz has some nice plugins coming to what timer source you want to use, so if there's a better way to get a time, these should make things easy to add.
These plugins can do no magic, if the hardware timer is slow, the whole thing gets slow.
Quoted 6 lines
Quoted 3 lines
- calculations: jiffies is a long integer whereas
ktime_tis 64bit, so if you need a lot of complex time calculation, you should take the cost for 32bit archs into account.Again, the cost is negligible for ms resolution. And we're getting close to 2038 so we need to be thinking in 64 bits anyway ;)
That one is not really an issue in the kernel, unless you want an uptime of a hundred years.
Quoted 8 lines
Quoted 3 lines
- resolution: how precise must the timer be? jiffies can't represent time values less than 1ms, but if time is e.g. measured in 10th of a second, jiffies may be enough.
And they would be if that is all you need. But coming from an embedded point of view, that is not nearly enough. I really see HighRes making it into the kernel soon, and any new code in this area really needs to take that into account.
I'm not against HR timer, I have a problem with using them as timer for everything.
Quoted 5 lines
Quoted 2 lines
- timer life time: if only a short interval is needed (e.g. a fraction of a second)
timer_listis often a lot cheaper.And again, you are only limited to 1000 choices to go off in that fraction of a second if jiffies is the resolution (with jiffies an 1000HZ).
The point is still valid, short interval timer are cheaper using normal timer, independent of whether they are removed or they expire.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 5 lines · Roman Zippel
Hi,
Quoted 1 line
As for portablility, I believe John Stultz has some nice plugins coming to what timer source you want to use, so if there's a better way to get a time, these should make things easy to add.
These plugins can do no magic, if the hardware timer is slow, the whole thing gets slow.
The point is that you could switch both the timer and timeout implementations to jiffies if you wanted to, at the expense of the accuracy that a lot of people care about.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
- resolution: how precise must the timer be? jiffies can't
represent time values less than 1ms, but if time is e.g. measured in 10th of a second, jiffies may be enough.And they would be if that is all you need. But coming from an embedded point of view, that is not nearly enough. I really see HighRes making it into the kernel soon, and any new code in this area really needs to take that into account.
I'm not against HR timer, I have a problem with using them as timer for everything.
This is _exactly_ why there is the timer/timeout distinction. Some things don't care, and as a result use a timer wheel exactly like they always have. For the things that do, however, the new timer API provides it using the fastest hardware interface available.
Cheers,
Kyle Moffett
--
I didn't say it would work as a defense, just that they can spin that out for years in court if it came to it.
-- Rob Landley
Re: [patch 00/43] ktimer reworked
Quoted 7 lines · Kyle Moffett
Quoted 6 lines · Roman Zippel
Hi,
Quoted 3 lines
As for portablility, I believe John Stultz has some nice plugins coming to what timer source you want to use, so if there's a better way to get a time, these should make things easy to add.
Quoted 6 lines
Quoted 2 lines
These plugins can do no magic, if the hardware timer is slow, the whole thing gets slow.
The point is that you could switch both the timer and timeout implementations to jiffies if you wanted to, at the expense of the accuracy that a lot of people care about.
While I'm not challenging the possibility of doing this, my timekeeping work does not provide quite this level of flexibility you imply. Indeed one could use jiffies as a clocksource, limiting all time users (including ktimers) to jiffies resolution, but I would consider that to be abusing the interface.
thanks
-john
Re: [patch 00/43] ktimer reworked
In reply to Kyle MoffettHi,
Quoted 7 lines · Kyle Moffett
Quoted 2 lines
I'm not against HR timer, I have a problem with using them as timer for everything.
This is _exactly_ why there is the timer/timeout distinction. Some things don't care, and as a result use a timer wheel exactly like they always have. For the things that do, however, the new timer API provides it using the fastest hardware interface available.
This is about kernel programming - people should care. We have enough crap as it is. timer wheel is fast as well, but everything has its limits, putting this focus completely to delivery is nonsense. It can't be that difficult to put together a decent list of criteria, where to use which timer. Both are still _timer_, introducing this timer/timeout thing is only confusing.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 9 lines · Roman Zippel
Hi,
Quoted 4 lines · Kyle Moffett
Quoted 2 lines
I'm not against HR timer, I have a problem with using them as timer for everything.
This is _exactly_ why there is the timer/timeout distinction. Some things don't care, and as a result use a timer wheel exactly like they always have. For the things that do, however, the new timer API provides it using the fastest hardware interface available.
This is about kernel programming - people should care.
My _point_ is that some code doesn't care about accuracy. If a networking timeout occurs a half-second later than it should, nothing bad happens. We have configurable SCSI drive timeouts, precisely because it doesn't really matter all that much if we deliver it now or give the drive a couple seconds extra time to try to respond before signalling a reset. And I agree with you that people should care, this distinction is important.
Quoted 1 line
We have enough crap as it is. timer wheel is fast as well, but everything has its limits, putting this focus completely to delivery is nonsense. It can't be that difficult to put together a decent list of criteria, where to use which timer.
A ktimer should be used where the common case is the timer being added and expiring. A ktimeout should be used where the common case is the timer being added and removed before it expires. Simple enough?
Cheers,
Kyle Moffett
--
I lost interest in "blade servers" when I found they didn't throw knives at people who weren't supposed to be in your machine room.
-- Anthony de Boer
Re: [patch 00/43] ktimer reworked
Hi,
Quoted 1 line · Kyle Moffett
My _point_ is that some code doesn't care about accuracy.
That's not how it works, the timer wheel is accurate within its resolution.
Quoted 3 lines
A ktimer should be used where the common case is the timer being added and expiring. A ktimeout should be used where the common case is the timer being added and removed before it expires. Simple enough?
As I said before there are other more important criteria. Check my mail to Steven, I'm not repeating it here.
bye, Roman
Re: [patch 00/43] ktimer reworked
In reply to Roman ZippelQuoted 8 lines · Roman Zippel
Hi,
Quoted 3 lines
As for portablility, I believe John Stultz has some nice plugins coming to what timer source you want to use, so if there's a better way to get a time, these should make things easy to add.
These plugins can do no magic, if the hardware timer is slow, the whole thing gets slow.
I was stating that if you have a faster timer, you can switch to it without suffering much portability problems. Anyway, you can turn off HR if you don't need it and you still just get jiffy resolution. And we still lose the overhead of the timer wheel having a lot of timers that will expire.
...
Quoted 7 lines
Quoted 4 lines
And they would be if that is all you need. But coming from an embedded point of view, that is not nearly enough. I really see HighRes making it into the kernel soon, and any new code in this area really needs to take that into account.
I'm not against HR timer, I have a problem with using them as timer for everything.
When do we know to use HR or not? So do you think that adding HR should be specific to certain areas?
So now we could have three separate timers :)
expire_timer - A timer that is expected to expire nonexpire_timer - A timer not expected to expire (timeout) precision_timer - A timer that will expire at a specific high
resolution time.
This would clear things up a bit, but I don't think Andrew would go for three different timer interfaces ;) But it would give you what you and I both want.
1) a timer that won't use high resolution timers but is still efficient for the add->expire case.
2) a timer that will most likely not expire and is efficient for the add->remove case.
3) a timer that will expire but will use the slower but more precise hardware timer.
If this didn't cause more confusion for developers not knowing which timer to use, I would say this would be a good idea.
Actually, we could make a single API for this and the default being the nonexpire_timer (timeout). Just add a flag field that would tell it to use the expire or precision timer. Have the guts of the API be implemented separately.
Just a suggestion.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
- timer life time: if only a short interval is needed (e.g. a fraction of a second)
timer_listis often a lot cheaper.And again, you are only limited to 1000 choices to go off in that fraction of a second if jiffies is the resolution (with jiffies an 1000HZ).
The point is still valid, short interval timer are cheaper using normal timer, independent of whether they are removed or they expire.
As long as they don't need to be rehashed.
-- Steve
Re: [patch 00/43] ktimer reworked
In reply to Steven RostedtQuoted 14 lines · Steven Rostedt
Quoted 3 lines · Roman Zippel
Hi,
On Thu, 1 Dec 2005, Russell King wrote:
...
Quoted 9 lines
Quoted 3 lines
Hence, timers have the implication that they are _expected_ to expire. Timeouts have the implication that their expiry is an exceptional condition.
IOW a timeout uses a timer to implement an exceptional condition after a period of time expires.
Quoted 1 line
So can we stop rehashing this stupid discussion?
Quoted 8 lines
Quoted 6 lines
The naming isn't actually my primary concern. I want a precise definition of the expected behaviour and usage of the old and new timer system. If I had this, it would be far easier to choose a proper name. E.g. I still don't know why ktimeout should be restricted to raise just "error conditions", as the name implies.
ktimeout may not need to be restricted to anything.
But does it make sense to use it in any other circumstances? It sounds
like the rb-tree based ktimer system is suitable for the general case. So you can have a simple rule: use ktimeout for timing out when an expected event doesn't occur, and ktimer for everything else. Are there any situations where you want a timer optimized for the removal case that is not also monotonic and low-res? And are there any situations in practice other than the "timeout" one where you'd want to use a timer wheel instead of a rb-tree?
It sounds to me that the ktimer should be the general case, leaving
ktimeout to be optimized for one particular case (by e.g. decreasing the resolution to reduce cascades).
Andrew Wade
Re: [patch 00/43] ktimer reworked
Hi,
Quoted 9 lines · Andrew James Wade
But does it make sense to use it in any other circumstances? It sounds
like the rb-tree based ktimer system is suitable for the general case. So you can have a simple rule: use ktimeout for timing out when an expected event doesn't occur, and ktimer for everything else. Are there any situations where you want a timer optimized for the removal case that is not also monotonic and low-res? And are there any situations in practice other than the "timeout" one where you'd want to use a timer wheel instead of a rb-tree?
Quoted 3 lines
It sounds to me that the ktimer should be the general case, leaving
ktimeout to be optimized for one particular case (by e.g. decreasing the resolution to reduce cascades).
By reducing the resolution you only reduce the frequency of the cascading, but the amount of timer in the timer wheel at any time is still the same. So in general you will have less number of cascades, but not generally smaller cascades. The latter depends on the actual timeout value and its distribution over the wheel. This means you can tune the resolution to avoid most cascading for a specific situation, but that would be a rather bad general solution.
rbtree based timer are also not necessarily the better general case. The timer wheel still scales better with O(n) compared to the rbtree with O(n*log(n)).
It's really better to keep the focus of the new timer at high resolution timer, that's what it's really better at and we shouldn't try to use it for everything only because it has such a kool name.
bye, Roman
Re: [patch 00/43] ktimer reworked
Quoted 2 lines · Roman Zippel
...
rbtree based timer are also not necessarily the better general case. ...
... As you've mentioned before. Somehow I missed that. Thank you for your patience.
Andrew
Re: [patch 00/43] ktimer reworked
In reply to Russell KingQuoted 11 lines · Russell King
rmk, also a native English speaker, agrees with Ray, Thomas and Ingo.
As does dictionary.reference.com's definitions of timeout and timer:timeout
A period of time after which an error condition is raised if some event has not occured. A common example is sending a message. If the receiver does not acknowledge the message within some preset timeout period, a transmission error is assumed to have occured.
Quoted 7 lines
timer
a timepiece that measures a time interval and signals its end
Hence, timers have the implication that they are _expected_ to expire. Timeouts have the implication that their expiry is an exceptional condition.
Well timer_lists get around the problem quite neatly by handling both situations. In a way which has been learned by thousands of developers over many years.
The whole concept of separating "timers" from "timeouts" seems a step
backward to me. A large one. Why was it done, and can it be undone?
Re: [patch 00/43] ktimer reworked
Quoted 2 lines · Andrew Morton
The whole concept of separating "timers" from "timeouts" seems a step backward to me. A large one. Why was it done, and can it be undone?
i had a very similar opinion when i first talked to Thomas about HRES timers a couple of months ago. I told him that the only sane way to add HRES timers to the -rt tree is to integrate them into the existing timer wheel, and to avoid a duality of APIs. I told him that adding a separate HRES implementation is pretty much a 'non-starter'.
then many months of experimentation followed. We (well, Thomas mostly) patiently tried a sub-jiffy method, a split-lists method, all sorts of ways to merge high-res timers into the timer wheel. We got HRES timers work in every such design, but it looked way too ugly and had bad performance and latencies - and we wanted upsteam integration.
so after many months we realized that the core issue is that the requirements of 'timers' are unmixable with the requirements of 'timeouts'. See a more detailed analysis at:
http://lwn.net/Articles/152436/
i'll try to sum it up again very briefly: the timer wheel is a very well-optimized data structure geared towards 'timeout' type of use. But it is at the very edge of its 'feature reach', and we found no workable way to extend it into the directions we wanted to go. The moment we tried to extend it in one direction (e.g. to increase HZ to 1000000 to get 1 usec resolution), it started creaking in some other spots.
the only clean solution we found was to totally separate them, and to use the natural data structures for both of them: to keep the highly scalable timer wheel on the timeout side, and to use the slower but more flexible [and deterministic] timer trees on the timer side. [ktimers are still very fast - but they cannot possibly be as fast as the single list_add() of add_timer()!]. The two usage scenarios (timeouts and timers) do not care about each other.
we could merge the two by driving 'timeouts' via ktimers too - but there would be some unavoidable overhead to things like the TCP stack. But ktimers cannot be merged into timeouts, that's sure.
Ingo
Re: [patch 00/43] ktimer reworked
Quoted 4 lines · Ingo Molnar
we could merge the two by driving 'timeouts' via ktimers too - but there
would be some unavoidable overhead to things like the TCP stack. But ktimers cannot be merged into timeouts, that's sure.
I think you guys have an advantage over me because you've been discussing and thinking about this terminology for months. IOW, your lips are moving but all I hear is blah, blah, blah ;)
For instance, when Kyle came out with his one-sentence description of timers versus timeouts, I thought he had them backwards. Only apparently he didn't.
So either it's all confusing, or I'm dumb, or both. I can evade investigation of that by claiming that we should seek something which is unconfusing to even dumb people.
We have timer_lists. But you say they don't suit precision timers. Fine. So why cannot we call the new precision timers something like "precision timers" and avoid this semantic confusion over timeouts versus timers?
IOW: leave timer_lists alone. Just add the needed new subsystem and use it.
I guess old-timers can mentally do s/ktimeout/timer_list/ whenever they come across the danged thing, but it's a bit painful. If we called them "timer_list" and "hrtimer", things would be much clearer. Plus that's a description of what they *are*, rather than of how we expect them to be applied.
Re: [patch 00/43] ktimer reworked
Quoted 8 lines · Andrew Morton
Quoted 1 line · Ingo Molnar
we could merge the two by driving 'timeouts' via ktimers too - but there would be some unavoidable overhead to things like the TCP stack. But ktimers cannot be merged into timeouts, that's sure.
I think you guys have an advantage over me because you've been discussing and thinking about this terminology for months. IOW, your lips are moving but all I hear is blah, blah, blah ;)
For instance, when Kyle came out with his one-sentence description of timers versus timeouts, I thought he had them backwards. Only apparently he didn't.
So either it's all confusing, or I'm dumb, or both. I can evade investigation of that by claiming that we should seek something which is unconfusing to even dumb people.
I think part of this confusion was the little flamewar over naming. I think I can provide a succinct and simple description of this stuff, possibly as the start of some documentation:
In this patch there are two ways of setting up code to run at some point in the future: timers and timeouts.
A timeout (like waiting for somebody to answer the phone) is optimized to never happen (they will hopefully pick up first). If everything works perfectly; it will be stopped before it has a chance to go off.
A timer (like a kitchen timer telling you the cookies are done) is optimized to be added and sit around until it expires. You just don't turn off the timer and take the cookies out before they are done.
For the most part, you don't really care much about accuracy with a timeout. It needs to happen no earlier than the specified time, but if it occurs a second late, so what? The person might sit around waiting an extra few seconds for their friend to pick up, but that's not a major issue. On the other hand, you really *do* care about how accurate your timer is. If you wait an extra minute or two after the timer goes off before pulling the cookies from the oven, you have some rather inedible cookies.
Quoted 1 line
IOW: leave
timer_listsalone. Just add the needed new subsystem and use it.
I think that this is a relatively useful distinction to make, and perhaps we _should_ rename the subsystem and educate developers about the difference between timers and timeouts.
Cheers,
Kyle Moffett
--
Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
-- Brian Kernighan
Re: [patch 00/43] ktimer reworked
Quoted 11 lines · Kyle Moffett
In this patch there are two ways of setting up code to run at some point in the future: timers and timeouts.
A timeout (like waiting for somebody to answer the phone) is optimized to never happen (they will hopefully pick up first). If everything works perfectly; it will be stopped before it has a chance to go off.
A timer (like a kitchen timer telling you the cookies are done) is optimized to be added and sit around until it expires. You just don't turn off the timer and take the cookies out before they are done.
Heh, in my dumb non-native speaker mind I'd expectit the other way around, as in a timeout is expected to time out :) and a timer is expect to happen, as in say the timer the tells you your breakfast egg is ready.
Re: [patch 00/43] ktimer reworked
Quoted 9 lines · Christoph Hellwig
Quoted 8 lines · Kyle Moffett
In this patch there are two ways of setting up code to run at some point in the future: timers and timeouts.
A timeout (like waiting for somebody to answer the phone) is optimized to never happen (they will hopefully pick up first). If everything works perfectly; it will be stopped before it has a chance to go off.
Quoted 7 lines
Quoted 3 lines
A timer (like a kitchen timer telling you the cookies are done) is optimized to be added and sit around until it expires. You just don't turn off the timer and take the cookies out before they are done.
Heh, in my dumb non-native speaker mind I'd expectit the other way around, as in a timeout is expected to time out :) and a timer is expect to happen, as in say the timer the tells you your breakfast egg is ready.
Which is perfectly the point Kyle made.
The timer tells you that the cookies or your breakfast eggs are well done. You put them out of the oven or the pot at exactly the time when the timer event happens. You won't turn off the timer before your cookies/eggs are done, because you want them well done.
The timeout you set up is to remind you to switch off the oven before your kitchen starts to burn. This timeout is likely - not sure in your personal case :) - to be cancelled because you did think about switching off the oven in time. If you forgot it, it is not a big difference if you get the reminder 5 minutes earlier or later.
In case of the egg / cookie timer the distcintion of 1 to 5 minutes makes a big difference.
tglx
Re: [patch 00/43] ktimer reworked
Quoted 4 lines · Thomas Gleixner
Quoted 1 line · Christoph Hellwig
Heh, in my dumb non-native speaker mind I'd expectit the other way around, as in a timeout is expected to time out :) and a timer is expect to happen, as in say the timer the tells you your breakfast egg is ready.
Which is perfectly the point Kyle made.
In any case, the real important note here is that the two are pretty different concepts, ones that lend themselves to _very_ different optimizations, that are currently lumped together. The very fact that some developers easily get them confused says that we need a good clean implementation of both distinct APIs with comparable documentation, including a bunch of good example usages.
Cheers,
Kyle Moffett
--
I have yet to see any problem, however complicated, which, when you looked at it in the right way, did not become still more complicated.
-- Poul Anderson
Re: [patch 00/43] ktimer reworked
Quoted 8 lines · Kyle Moffett
Quoted 6 lines · Thomas Gleixner
Quoted 4 lines · Christoph Hellwig
Heh, in my dumb non-native speaker mind I'd expectit the other way around, as in a timeout is expected to time out :) and a timer is expect to happen, as in say the timer the tells you your breakfast egg is ready.
Quoted 9 lines
Quoted 1 line
Which is perfectly the point Kyle made.
In any case, the real important note here is that the two are pretty different concepts, ones that lend themselves to _very_ different optimizations, that are currently lumped together. The very fact that some developers easily get them confused says that we need a good clean implementation of both distinct APIs with comparable documentation, including a bunch of good example usages.
Or just leave the timer_lists as they are.
If I'm going to spend the next two years buried in helpful s/timer_list/ktimeout/ patches then there'd better be a darn good reason for the rename, thanks. I don't see one.
Re: [patch 00/43] ktimer reworked
In reply to Kyle MoffettQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Kyle" == Kyle Moffett <[email protected]> writes:
Quoted 7 lines · Thomas Gleixner
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines
Heh, in my dumb non-native speaker mind I'd expectit the other way around, as in a timeout is expected to time out :) and a timer is expect to happen, as in say the timer the tells you your breakfast egg is ready.
Which is perfectly the point Kyle made.
Kyle> In any case, the real important note here is that the two are Kyle> pretty different concepts, ones that lend themselves to _very_ Kyle> different optimizations, that are currently lumped together. Kyle> The very fact that some developers easily get them confused says Kyle> that we need a good clean implementation of both distinct APIs Kyle> with comparable documentation, including a bunch of good example Kyle> usages.
I think the problem is in using the work 'time' in both. Split that so that they are seperate, and alot of the confusion will go away. Do I have a usefull suggestion? No, I'm being fairly dumb this morning... but just seeing all your smart guys getting confused makes me think they rest of us would be lost too.
Hmm... how about:
timer - gotta let me know exactly when it expires, I won't
touch it until it does.
reminder - I'll generally clean this up before it fires ,
don't care if I get reminded a bit later.
John
Re: [patch 00/43] ktimer reworked
In reply to Thomas GleixnerQuoted 18 lines · Thomas Gleixner
Quoted 9 lines · Christoph Hellwig
Quoted 8 lines · Kyle Moffett
In this patch there are two ways of setting up code to run at some point in the future: timers and timeouts.
A timeout (like waiting for somebody to answer the phone) is optimized to never happen (they will hopefully pick up first). If everything works perfectly; it will be stopped before it has a chance to go off.
Quoted 7 lines
Quoted 3 lines
A timer (like a kitchen timer telling you the cookies are done) is optimized to be added and sit around until it expires. You just don't turn off the timer and take the cookies out before they are done.
Heh, in my dumb non-native speaker mind I'd expectit the other way around, as in a timeout is expected to time out :) and a timer is expect to happen, as in say the timer the tells you your breakfast egg is ready.
Quoted 6 lines
Which is perfectly the point Kyle made.
The timer tells you that the cookies or your breakfast eggs are well done. You put them out of the oven or the pot at exactly the time when the timer event happens. You won't turn off the timer before your cookies/eggs are done, because you want them well done.
As I mentioned before, this is where you expect an event to happen (I want x to happen at some time). The event being to take out the cookies.
Quoted 6 lines
The timeout you set up is to remind you to switch off the oven before your kitchen starts to burn. This timeout is likely - not sure in your personal case :) - to be cancelled because you did think about switching off the oven in time. If you forgot it, it is not a big difference if you get the reminder 5 minutes earlier or later.
And this is where a time out is done when some event doesn't happen (I want to know if x didn't happen by some time). The event here is to turn off the oven and the time out is to let you know that you didn't and the house will soon burn down if you don't turn it off.
Quoted 2 lines
In case of the egg / cookie timer the distcintion of 1 to 5 minutes makes a big difference.
I started kernel programming with writing experimental network stacks. So to me "timeout" is strongly engraved in my head as to let you know when something didn't happen, and you need to do something to recover for it. If you send out a packet and you don't receive an ACK, you have a timeout let you know that it probably didn't make it (or the ACK didn't make it to you), and send another packet out. But most likely, (unless you are on a really unreliable network) you get your ACK and you can remove the time out.
I know that timeouts are used tremendously in the networking code, so separating them from precision timers would likely be a good thing.
-- Steve
Re: [patch 00/43] ktimer reworked
In reply to Kyle MoffettHi,
Quoted 7 lines · Kyle Moffett
A timeout (like waiting for somebody to answer the phone) is optimized to never happen (they will hopefully pick up first). If everything works perfectly; it will be stopped before it has a chance to go off.
A timer (like a kitchen timer telling you the cookies are done) is optimized to be added and sit around until it expires. You just don't turn off the timer and take the cookies out before they are done.
Making this the primary criteria for choosing a timer system would be a huge mistake. As I wrote in a previous mail there are other, more important criteria.
So far I still thought this was about kernel programming and not about Aunt Tillies cooking show. Can we please bring this back to a technical level?
bye, Roman
Re: [patch 00/43] ktimer reworked
In reply to Ray LeeHi,
Quoted 5 lines · Ray Lee
Actually, no you won't. The athlete will say "A timeout? Something out of the ordinary happened, and coach wants me to go to the sidelines to talk." Timeouts are unexpected and exceptional, whether you're an athlete or a piece of code. On the other hand, they have a timer that everyone *expects* to expire at the end of the quarter or game.
Please be precise, there is of course a common base, but it's not the same. In sports a timeout is the actual event that interrupts something. In code it's a time _period_ until an exceptional event. A timer delivers an asynchronous event after a specified timeout, so they're always "unexpected and exceptional". (You can of course constantly poll the timer to make it a little less unexpected, but then you don't really need to set a timer in first place.)
bye, Roman