timekeeping: get_jiffies_boot_64() for jiffies that include sleep time
Subsystems (2)
PATCH v2 1/1 timekeeping: get_jiffies_boot_64() for jiffies that include sleep time
This enables using the usual get_jiffies_64() but taking into account time spent sleeping, giving the high performance characteristics of querying jiffies without the drawback.
We accomplish this by precomputing the boottime jiffies offset whenever it is updated, rather than doing the expensive-ish div_u64 on each query.
Since the resolution of this is in terms of jiffies, this allows determining limits for comparison in terms of jiffies too, which makes the comparisons more exact, despite jiffies being a fairly coarse stamp.
Adding the suspend offset to jiffies as such doesn't actually race in a way different from the usual races associated with the suspend offset:
either boot offset has been updated before the call to get_jiffies_boot_64(), in which case we're fine, or it hasn't in which case, this is no different than any of the existing suspend querying functions, which may be invoked early in system resumption before the offset is updated.
Cc; John Stultz <[email protected]>
--- include/linux/jiffies.h | 1 + include/linux/timekeeper_internal.h | 2 ++ kernel/time/timekeeping.c | 11 +++++++++++ 3 files changed, 14 insertions(+)
include/linux/jiffies.h +1 -0
@@ -80,6 +80,7 @@ extern int register_refined_jiffies(long clock_tick_rate); extern u64 __cacheline_aligned_in_smp jiffies_64; extern unsigned long volatile __cacheline_aligned_in_smp __jiffy_arch_data jiffies; +u64 get_jiffies_boot_64 (void );#if (BITS_PER_LONG <64 )u64 get_jiffies_64 (void );#else
include/linux/timekeeper_internal.h +2 -0
@@ -51,6 +51,7 @@ struct tk_read_base {* @wall_to_monotonic: CLOCK_REALTIME to CLOCK_MONOTONIC offset * @offs_real: Offset clock monotonic -> clock realtime * @offs_boot: Offset clock monotonic -> clock boottime +* @offs_boot_jiffies64 Offset clock monotonic -> clock boottime in jiffies64 * @offs_tai: Offset clock monotonic -> clock tai * @tai_offset: The current UTC to TAI offset in seconds * @clock_was_set_seq: The sequence number of clock was set events
@@ -93,6 +94,7 @@ struct timekeeper {struct timespec64 wall_to_monotonic;ktime_t offs_real;ktime_t offs_boot; +u64 offs_boot_jiffies64;ktime_t offs_tai;s32 tai_offset;unsigned int clock_was_set_seq;
kernel/time/timekeeping.c +11 -0
@@ -146,6 +146,7 @@ static void tk_set_wall_to_mono(struct timekeeper *tk, struct timespec64 wtm)static inline void tk_update_sleep_time (struct timekeeper *tk,ktime_t delta) { tk->offs_boot =ktime_add (tk->offs_boot, delta); + tk->offs_boot_jiffies64 =nsecs_to_jiffies64 (ktime_to_ns (tk->offs_boot)); }/*
@@ -539,6 +540,16 @@ u64 ktime_get_real_fast_ns(void) }EXPORT_SYMBOL_GPL (ktime_get_real_fast_ns); +/** +* get_jiffies_boot_64 - The normal get_jiffies_64(), but taking into +* account the time spent sleeping. +*/ +u64 get_jiffies_boot_64 (void ) +{ +return get_jiffies_64 () + tk_core.timekeeper.offs_boot_jiffies64; +} +EXPORT_SYMBOL (get_jiffies_boot_64); +/** * halt_fast_timekeeper - Prevent fast timekeeper from accessing clocksource. * @tk: Timekeeper to snapshot.
Signature · 1 line
2.21.0
Quoted 4 lines · Jason A. Donenfeld
This enables using the usual
get_jiffies_64()but taking into account time spent sleeping, giving the high performance characteristics of querying jiffies without the drawback.
Can you quantify how much this gains you over ktime_get_coarse_boottime in practice? You are effectively adding yet another abstraction for time, which is something I'd hope to avoid unless you have a strong reason other than it being faster in theory.
How often do you read the current time?
Arnd
Hi Arnd,
Quoted 4 lines · Arnd Bergmann
Can you quantify how much this gains you over
ktime_get_coarse_boottimein practice? You are effectively adding yet another abstraction for time, which is something I'd hope to avoid unless you have a strong reason other than it being faster in theory.
Excellent idea. It turns out to be precisely 0 (see below). A motivation still remains, though: this allows comparison with units specified in terms of jiffies, which means that the unit being compared matches the exact tick of the clock, making those comparisons as precise as possible, for what they are. I suppose you could argue, on the other hand, that nanoseconds give so much precision already, that approximations using them amount practically to the same thing. I'm not sure which way to reason about that.
For interest, here are a few comparisons taken with kbench9000:
get_jiffies_boot_64 26ktime_get_coarse_boottime 26ktime_get_boot_fast_ns with tsc 70ktime_get_boot_fast_ns with hpet 4922ktime_get_boot_fast_ns with acpi_pm 1884
As expected, hpet is really quite painful.
Jason
Quoted 20 lines · Jason A. Donenfeld
Hi Arnd,
Quoted 4 lines · Arnd Bergmann
Can you quantify how much this gains you over
ktime_get_coarse_boottimein practice? You are effectively adding yet another abstraction for time, which is something I'd hope to avoid unless you have a strong reason other than it being faster in theory.Excellent idea. It turns out to be precisely 0 (see below). A motivation still remains, though: this allows comparison with units specified in terms of jiffies, which means that the unit being compared matches the exact tick of the clock, making those comparisons as precise as possible, for what they are. I suppose you could argue, on the other hand, that nanoseconds give so much precision already, that approximations using them amount practically to the same thing. I'm not sure which way to reason about that.
For interest, here are a few comparisons taken with kbench9000:
Quoted 7 lines
get_jiffies_boot_6426ktime_get_coarse_boottime26ktime_get_boot_fast_nswith tsc 70ktime_get_boot_fast_nswith hpet 4922ktime_get_boot_fast_nswithacpi_pm1884As expected, hpet is really quite painful.
I would prefer not to add the new interface then. We might in fact move users of get_jiffies_64() to ktime_get_coarse() for consistency given the small overhead of that function.
Arnd
Quoted 11 lines · Arnd Bergmann
Quoted 7 lines
get_jiffies_boot_6426ktime_get_coarse_boottime26ktime_get_boot_fast_nswith tsc 70ktime_get_boot_fast_nswith hpet 4922ktime_get_boot_fast_nswithacpi_pm1884As expected, hpet is really quite painful.
I would prefer not to add the new interface then. We might in fact move users of
get_jiffies_64()toktime_get_coarse()for consistency given the small overhead of that function.
In light of the measurements, that seems like a good plan to me.
One thing to consider with moving jiffies users over that way is ktime_t. Do you want to introduce helpers like ktime_get_boot_coarse_ns(), just like there is already with the other various functions like ktime_get_boot_ns(), ktime_get_boot_fast_ns(), etc? (I'd personally prefer using the _ns variants, at least.) I can send a patch for this.
Jason
Quoted 14 lines · Jason A. Donenfeld
Quoted 11 lines · Arnd Bergmann
Quoted 7 lines
get_jiffies_boot_6426ktime_get_coarse_boottime26ktime_get_boot_fast_nswith tsc 70ktime_get_boot_fast_nswith hpet 4922ktime_get_boot_fast_nswithacpi_pm1884As expected, hpet is really quite painful.
I would prefer not to add the new interface then. We might in fact move users of
get_jiffies_64()toktime_get_coarse()for consistency given the small overhead of that function.
Quoted 8 lines
In light of the measurements, that seems like a good plan to me.
One thing to consider with moving jiffies users over that way is
ktime_t. Do you want to introduce helpers likektime_get_boot_coarse_ns(), just like there is already with the other various functions likektime_get_boot_ns(),ktime_get_boot_fast_ns(), etc? (I'd personally prefer using the_nsvariants, at least.) I can send a patch for this.
That sounds reasonable, but then I think we should have the full set of coarse_*_ns() functions, again for consistency:
u64 ktime_get_coarse_ns(void) u64 ktime_get_coarse_boottime_ns(void) u64 ktime_get_coarse_real_ns(void) u64 ktime_get_coarse_clocktai_ns(void)
and document them in Documentation/core-api/timekeeping.rst.
We seem to also be lacking the basic ktime_get_coarse(), which seems like a major omission.
Both ktime_get_coarse_ns and ktime_get_coarse can be wrappers around ktime_get_coarse_ts64() then, while the others would use ktime_get_coarse_with_offset().
Arnd
Quoted 33 lines · Arnd Bergmann
Quoted 14 lines · Jason A. Donenfeld
Quoted 11 lines · Arnd Bergmann
Quoted 7 lines
get_jiffies_boot_6426ktime_get_coarse_boottime26ktime_get_boot_fast_nswith tsc 70ktime_get_boot_fast_nswith hpet 4922ktime_get_boot_fast_nswithacpi_pm1884As expected, hpet is really quite painful.
I would prefer not to add the new interface then. We might in fact move users of
get_jiffies_64()toktime_get_coarse()for consistency given the small overhead of that function.Quoted 8 lines
In light of the measurements, that seems like a good plan to me.
One thing to consider with moving jiffies users over that way is
ktime_t. Do you want to introduce helpers likektime_get_boot_coarse_ns(), just like there is already with the other various functions likektime_get_boot_ns(),ktime_get_boot_fast_ns(), etc? (I'd personally prefer using the_nsvariants, at least.) I can send a patch for this.That sounds reasonable, but then I think we should have the full set of coarse_*_ns() functions, again for consistency:
u64
ktime_get_coarse_ns(void)u64ktime_get_coarse_boottime_ns(void)u64ktime_get_coarse_real_ns(void)u64ktime_get_coarse_clocktai_ns(void)
Quoted 7 lines
and document them in
Documentation/core-api/timekeeping.rst.We seem to also be lacking the basic
ktime_get_coarse(), which seems like a major omission.
Bothktime_get_coarse_nsandktime_get_coarsecan be wrappers aroundktime_get_coarse_ts64()then, while the others would usektime_get_coarse_with_offset().
Exactly what I had in mind. I'll have something posted fairly soon.
Jason
PATCH 1/3 timekeeping: add missing non-_ns functions for fast accessors
Previously there was no analogue to get proper ktime_t versions of the fast variety of ktime invocations. This commit makes the interface uniform with the other accessors.
--- Documentation/core-api/timekeeping.rst | 7 +++- include/linux/timekeeping.h | 28 ++++++++++++-- kernel/time/timekeeping.c | 52 +++++++++++++------------- 3 files changed, 55 insertions(+), 32 deletions(-)
Documentation/core-api/timekeeping.rst +6 -1
@@ -94,7 +94,7 @@ different format depending on what is required by the user: down the seconds to the full seconds of the last timer tick using the respective reference. -Coarse and fast_ns access +Coarse and fast access ------------------------- Some additional variants exist for more specialized cases:
@@ -125,6 +125,11 @@ Some additional variants exist for more specialized cases: up to several microseconds on older hardware with an external clocksource. +.. c:function:: ktime_t ktime_get_mono_fast_ns( void ) + ktime_t ktime_get_raw_fast_ns( void ) + ktime_t ktime_get_boottime_fast_ns( void ) + ktime_t ktime_get_real_fast_ns( void ) + .. c:function:: u64 ktime_get_mono_fast_ns( void ) u64 ktime_get_raw_fast_ns( void ) u64 ktime_get_boot_fast_ns( void )
include/linux/timekeeping.h +24 -4
@@ -146,10 +146,30 @@ static inline u64 ktime_get_raw_ns(void)return ktime_to_ns (ktime_get_raw ()); } -extern u64 ktime_get_mono_fast_ns (void ); -extern u64 ktime_get_raw_fast_ns (void ); -extern u64 ktime_get_boot_fast_ns (void ); -extern u64 ktime_get_real_fast_ns (void ); +extern ktime_t ktime_get_mono_fast (void ); +extern ktime_t ktime_get_raw_fast (void ); +extern ktime_t ktime_get_boottime_fast (void ); +extern ktime_t ktime_get_real_fast (void ); + +static inline u64 ktime_get_mono_fast_ns (void ) +{ +return ktime_to_ns (ktime_get_mono_fast ()); +} + +static inline u64 ktime_get_raw_fast_ns (void ) +{ +return ktime_to_ns (ktime_get_raw_fast ()); +} + +static inline u64 ktime_get_boot_fast_ns (void ) +{ +return ktime_to_ns (ktime_get_boottime_fast ()); +} + +static inline u64 ktime_get_real_fast_ns (void ) +{ +return ktime_to_ns (ktime_get_real_fast ()); +}/* * timespec64/time64_t interfaces utilizing the ktime based ones
kernel/time/timekeeping.c +25 -27
113 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH 2/3 timekeeping: use proper ktime_add when adding nsecs in coarse offset
While this doesn't actually amount to a real difference, since the macro evaluates to the same thing, every place else operates on ktime_t using these functions, so let's not break the pattern.
--- kernel/time/timekeeping.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/time/timekeeping.c +1 -1
@@ -817,7 +817,7 @@ ktime_t ktime_get_coarse_with_offset(enum tk_offsets offs) }while (read_seqcount_retry (&tk_core.seq, seq)); -return base + nsecs; +return ktime_add_ns (base, nsecs); }EXPORT_SYMBOL_GPL (ktime_get_coarse_with_offset);
Signature · 1 line
2.21.0
Re: [PATCH 2/3] timekeeping: use proper ktime_add when adding nsecs in coarse offset
Quoted 9 lines · Jason A. Donenfeld
While this doesn't actually amount to a real difference, since the macro evaluates to the same thing, every place else operates on
ktime_tusing these functions, so let's not break the pattern.Fixes e3ff9c3678b4 ("timekeeping: Repair ktime_get_coarse*() granularity")Signed-off-by Jason A. Donenfeld <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Thomas Gleixner <[email protected]>
PATCH 3/3 timekeeping: add missing _ns functions for coarse accessors
In reply to Jason A. DonenfeldThis further unifies the accessors for the fast and coarse functions, so that the same types of functions are available for each. There was also a bit of confusion with the documentation, which prior advertised a function that has never existed. Finally, the vanilla ktime_get_coarse() was omitted from the API originally, so this fills this oversight.
--- Documentation/core-api/timekeeping.rst | 10 +++++++--- include/linux/timekeeping.h | 27 ++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 3 deletions(-)
Documentation/core-api/timekeeping.rst +7 -3
@@ -99,16 +99,20 @@ Coarse and fast access Some additional variants exist for more specialized cases: -.. c:function:: ktime_t ktime_get_coarse_boottime( void ) +.. c:function:: ktime_t ktime_get_coarse( void ) + ktime_t ktime_get_coarse_boottime( void ) ktime_t ktime_get_coarse_real( void ) ktime_t ktime_get_coarse_clocktai( void ) - ktime_t ktime_get_coarse_raw( void ) + +.. c:function:: u64 ktime_get_coarse_ns( void ) + u64 ktime_get_boot_coarse_ns( void ) + u64 ktime_get_real_coarse_ns( void ) + u64 ktime_get_tai_coarse_ns( void ) .. c:function:: void ktime_get_coarse_ts64( struct timespec64 * ) void ktime_get_coarse_boottime_ts64( struct timespec64 * ) void ktime_get_coarse_real_ts64( struct timespec64 * ) void ktime_get_coarse_clocktai_ts64( struct timespec64 * ) - void ktime_get_coarse_raw_ts64( struct timespec64 * ) These are quicker than the non-coarse versions, but less accurate, corresponding to CLOCK_MONONOTNIC_COARSE and CLOCK_REALTIME_COARSE
include/linux/timekeeping.h +27 -0
@@ -113,6 +113,33 @@ static inline ktime_t ktime_get_coarse_clocktai(void)return ktime_get_coarse_with_offset (TK_OFFS_TAI ); } +static inline ktime_t ktime_get_coarse (void ) +{ +struct timespec64 ts; +ktime_get_coarse_ts64 (&ts); +return timespec64_to_ktime (ts); +} + +static inline u64 ktime_get_coarse_ns (void ) +{ +return ktime_to_ns (ktime_get_coarse ()); +} + +static inline u64 ktime_get_real_coarse_ns (void ) +{ +return ktime_to_ns (ktime_get_coarse_real ()); +} + +static inline u64 ktime_get_boot_coarse_ns (void ) +{ +return ktime_to_ns (ktime_get_coarse_boottime ()); +} + +static inline u64 ktime_get_tai_coarse_ns (void ) +{ +return ktime_to_ns (ktime_get_coarse_clocktai ()); +} +/** * ktime_mono_to_real - Convert monotonic time to clock realtime */
Signature · 1 line
2.21.0
Re: [PATCH 3/3] timekeeping: add missing _ns functions for coarse accessors
Quoted 11 lines · Jason A. Donenfeld
Documentation/core-api/timekeeping.rst +0
-1@@ -99,16 +99,20 @@ Coarse and fast access Some additional variants exist for more specialized cases: -.. c:function:: ktime_t ktime_get_coarse_boottime( void )
Documentation/core-api/timekeeping.rst Coarse and fast access · line 110
Quoted 10 lines
Documentation/core-api/timekeeping.rst +7
-1+.. c:function:: ktime_t ktime_get_coarse( void ) + ktime_t ktime_get_coarse_boottime( void ) ktime_t ktime_get_coarse_real( void ) ktime_t ktime_get_coarse_clocktai( void ) - ktime_t ktime_get_coarse_raw( void ) + +.. c:function:: u64 ktime_get_coarse_ns( void ) + u64 ktime_get_boot_coarse_ns( void ) + u64 ktime_get_real_coarse_ns( void ) + u64 ktime_get_tai_coarse_ns( void )
I would prefer the 'coarse' on the other side, i.e. ktime_get_coarse_real_ns instead of ktime_get_real_coarse_ns, as this is what we already have with ktime_get_coarse_real_ts64.
I originally went with that order to avoid the function sounding "real coarse", although I have to admit that it was before Thomas fixed it in e3ff9c3678b4 ("timekeeping: Repair ktime_get_coarse*() granularity"). ;-)
I would also prefer _boottime over _boot. Unfortunately we are already inconsistent and have roughly the same number of callers for ktime_get_boot_ns() and ktime_get_boottime().
Arnd
Re: [PATCH 3/3] timekeeping: add missing _ns functions for coarse accessors
Quoted 8 lines · Arnd Bergmann
I would prefer the 'coarse' on the other side, i.e.
ktime_get_coarse_real_nsinstead ofktime_get_real_coarse_ns, as this is what we already have withktime_get_coarse_real_ts64.I originally went with that order to avoid the function sounding "real coarse", although I have to admit that it was before Thomas fixed it in
e3ff9c3678b4("timekeeping: Repair ktime_get_coarse*() granularity"). ;-)
I can do this, but that means also I'll change get_real_fast to get_fast_real, too, in order to be consistent. Is that okay?
Re: [PATCH 3/3] timekeeping: add missing _ns functions for coarse accessors
Quoted 5 lines · Jason A. Donenfeld
Quoted 4 lines · Arnd Bergmann
I would prefer the 'coarse' on the other side, i.e.
ktime_get_coarse_real_nsinstead ofktime_get_real_coarse_ns, as this is what we already have withktime_get_coarse_real_ts64.
Quoted 7 lines
Quoted 4 lines
I originally went with that order to avoid the function sounding "real coarse", although I have to admit that it was before Thomas fixed it in
e3ff9c3678b4("timekeeping: Repair ktime_get_coarse*() granularity"). ;-)I can do this, but that means also I'll change
get_real_fasttoget_fast_real, too, in order to be consistent. Is that okay?
I care less about these since ktime_get_real_fast_ns() already exists. My preference would be leaving alons the _fast_ns() functions for now, but making everything else consistent instead.
Thomas created the _fast_ns() accessors with a specific application in mind, and I suppose we don't really want them to be used much beyond that. I wonder if we should try to come up with a better name instead of "fast" that makes the purpose clearer and does not suggest that it's faster to read than the "coarse" version.
Arnd
Re: [PATCH 3/3] timekeeping: add missing _ns functions for coarse accessors
Quoted 9 lines · Arnd Bergmann
I care less about these since
ktime_get_real_fast_ns()already exists. My preference would be leaving alons the_fast_ns()functions for now, but making everything else consistent instead.Thomas created the
_fast_ns()accessors with a specific application in mind, and I suppose we don't really want them to be used much beyond that. I wonder if we should try to come up with a better name instead of "fast" that makes the purpose clearer and does not suggest that it's faster to read than the "coarse" version.
Oh shoot, I just submitted v3 having not seen this. Does v3's 4/4 look fine, or shall I undo the _fast switcheroo and resubmit?
Jason
Re: [PATCH 3/3] timekeeping: add missing _ns functions for coarse accessors
Quoted 6 lines · Jason A. Donenfeld
Quoted 4 lines · Arnd Bergmann
I care less about these since
ktime_get_real_fast_ns()already exists. My preference would be leaving alons the_fast_ns()functions for now, but making everything else consistent instead.
Quoted 8 lines
Quoted 5 lines
Thomas created the
_fast_ns()accessors with a specific application in mind, and I suppose we don't really want them to be used much beyond that. I wonder if we should try to come up with a better name instead of "fast" that makes the purpose clearer and does not suggest that it's faster to read than the "coarse" version.Oh shoot, I just submitted v3 having not seen this. Does v3's 4/4 look fine, or shall I undo the
_fastswitcheroo and resubmit?
I'd still prefer to leave out anything touching the _fast functions from patches 1 and 4. AFAICT, that would leave ktime_get_tai_ns() and ktime_get_boot_ns() to be renamed to clocktai() and bootime() respectively.
Arnd
Re: [PATCH 1/3] timekeeping: add missing non-_ns functions for fast accessors
In reply to Jason A. DonenfeldQuoted 8 lines · Jason A. Donenfeld
Previously there was no analogue to get proper
ktime_tversions of the fast variety of ktime invocations. This commit makes the interface uniform with the other accessors.Signed-off-by Jason A. Donenfeld <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Thomas Gleixner <[email protected]>
Consistent is good, not sure if there is a real use for the *_fast() functions returning a ktime_t, but I don't mind adding them.
Documentation/core-api/timekeeping.rst Some additional variants exist for more specialized cases: · line 135
Quoted 8 lines
+.. c:function::
ktime_tktime_get_mono_fast_ns( void )Documentation/core-api/timekeeping.rst +4
-0+ ktime_t ktime_get_raw_fast_ns( void ) + ktime_t ktime_get_boottime_fast_ns( void ) + ktime_t ktime_get_real_fast_ns( void ) + .. c:function:: u64 ktime_get_mono_fast_ns( void ) u64 ktime_get_raw_fast_ns( void ) u64 ktime_get_boot_fast_ns( void )
Typo: you have the same function names listed twice here, one of them should be ktime_get_mono_fast() instead of ktime_get_mono_fast_ns().
Also, we might want to rename ktime_get_boot_fast_ns() to ktime_get_boottime_fast_ns in the process. It seems there is only a single caller.
Arnd
Re: [PATCH 1/3] timekeeping: add missing non-_ns functions for fast accessors
Quoted 3 lines · Arnd Bergmann
Typo: you have the same function names listed twice here, one of them should be
ktime_get_mono_fast()instead ofktime_get_mono_fast_ns().
Nice catch. Vim twitches gone crazy.
Quoted 3 lines
Also, we might want to rename
ktime_get_boot_fast_ns()toktime_get_boottime_fast_nsin the process. It seems there is only a single caller.
And tai -> clocktai on the others. I can send a followup patch to unify all those after this set.
Re: [PATCH 1/3] timekeeping: add missing non-_ns functions for fast accessors
Quoted 6 lines · Jason A. Donenfeld
Quoted 3 lines · Arnd Bergmann
Typo: you have the same function names listed twice here, one of them should be
ktime_get_mono_fast()instead ofktime_get_mono_fast_ns().
Quoted 8 lines
Nice catch. Vim twitches gone crazy.
Quoted 3 lines
Also, we might want to rename
ktime_get_boot_fast_ns()toktime_get_boottime_fast_nsin the process. It seems there is only a single caller.And tai -> clocktai on the others. I can send a followup patch to unify all those after this set.
Yes, that's probably best.
Arnd