Unify interface to persistent CMOS/RTC/whatever clock
Subsystems (3)
Quoted 4 lines
Almost every arch has some form of persistent clock (CMOS, RTC, etc)
which is normally used at boot time to initialize xtime andwall_to_monotonic. As part of the timekeeping consolidation, I propose the following generic interface to the arch specific persistent clock:
Hmm, this seems to ignore the RTC framework rather entirely, which seems to me like the wrong implementation approach. You'd likely have noticed that if you had supported a few ARM boards. :)
Here's a fairly common scenario for an embedded system: the battery backed clock is accessed through I2C or SPI, rather than an ISA style direct register access, so it's not accessible until after those driver stacks have initialized.
Or similarly, the SOC family may have a powerful RTC ("arch specific") with alarm and system wakeup capabilities, but it may not be the system's battery backed clock ... so that RTC would be initialized from another one, which is accessed using I2C/SPI/etc. (RTCs integrated on SOCs evidently have a hard time being as power-miserly as the discrete ones.)
The approach in this patch doesn't seem to play well with those scenarios, because it expects to do timekeeping setup long before the system's RTC is necessarily going to be available ... and doesn't have an answer for those cases where the RTC is unavailable before e.g. late_initcall().
I'd be more interested in something that improves on CONFIG_RTC_HCTOSYS, and for example addresses the need to update the system wall time from such RTCs after resume, not just at boot time.
- Dave
Quoted 8 lines · David Brownell
Quoted 4 lines
Almost every arch has some form of persistent clock (CMOS, RTC, etc)
which is normally used at boot time to initialize xtime andwall_to_monotonic. As part of the timekeeping consolidation, I propose the following generic interface to the arch specific persistent clock:Hmm, this seems to ignore the RTC framework rather entirely, which seems to me like the wrong implementation approach. You'd likely have noticed that if you had supported a few ARM boards. :)
Good point. I've not spent enough time looking at the RTC code, and it looks like it has potential to resolve some of the issues I'm looking at. Although, how common is it really? Could we currently replace all arches xtime initialization code with it? I suspect that would require it to be built at all times. Would that be acceptable?
Quoted 4 lines
Here's a fairly common scenario for an embedded system: the battery backed clock is accessed through I2C or SPI, rather than an ISA style direct register access, so it's not accessible until after those driver stacks have initialized.
Yea. Although late initialization isn't that much of an issue in my mind.
Quoted 5 lines
Or similarly, the SOC family may have a powerful RTC ("arch specific") with alarm and system wakeup capabilities, but it may not be the system's battery backed clock ... so that RTC would be initialized from another one, which is accessed using I2C/SPI/etc. (RTCs integrated on SOCs evidently have a hard time being as power-miserly as the discrete ones.)
SOC?
Quoted 4 lines
The approach in this patch doesn't seem to play well with those scenarios, because it expects to do timekeeping setup long before the system's RTC is necessarily going to be available ... and doesn't have an answer for those cases where the RTC is unavailable before e.g.
late_initcall().
That's fair. While the patch I just sent didn't consider these cases, I don't think these scenarios break the intent of what I'm wanting to do.
Quoted 3 lines
I'd be more interested in something that improves on
CONFIG_RTC_HCTOSYS, and for example addresses the need to update the system wall time from such RTCs after resume, not just at boot time.
Agreed.
Basically what I'm trying to provide w/ read_persistent_clock() is:
1) A method for generic timekeeping to access a battery backed clock to avoid potential suspend/resume ordering problems.
2) A generic interface to a low-res hardware driven clock for the hangcheck-timer and other watchdog timers.
3) To cleanup the arch time initialization code and encapsulate xtime
Do you think the RTC interface is sufficient for this?
Thanks, I appreciate the feedback!
-john
Quoted 9 lines · john stultz
Quoted 4 lines · David Brownell
Hmm, this seems to ignore the RTC framework rather entirely, which seems to me like the wrong implementation approach. You'd likely have noticed that if you had supported a few ARM boards. :)
Good point. I've not spent enough time looking at the RTC code, and it looks like it has potential to resolve some of the issues I'm looking at. Although, how common is it really?
Increasingly so; it solves a lot of reinvent-the-wheel problems, much like the recent generic time-of-day updates are doing. And just like with clocksource updates, not all architectures have converted yet!
Quoted 2 lines
Could we currently replace all
arches xtime initialization code with it?
Not yet, I think, mostly because RTCs are still being converted to that framework.
For ACPI PCs, I converted drivers/char/rtc.c to that framework (see:
http://article.gmane.org/gmane.linux.kernel/427493) covering most hardware over the last few years.
Quoted 2 lines
I suspect that would require
it to be built at all times. Would that be acceptable?
It's just a question of _which_ code to build all the time, right? Having it be standardized core code would seem appropriate.
Quoted 7 lines
Quoted 4 lines
Here's a fairly common scenario for an embedded system: the battery backed clock is accessed through I2C or SPI, rather than an ISA style direct register access, so it's not accessible until after those driver stacks have initialized.
Yea. Although late initialization isn't that much of an issue in my mind.
... except that unless I traced the code wrong, your current patch is initializing the (wall) clock extremely early ... ergo my comment about assuming ISA style access. I'm happy to call that a minor implementation artifact though, since I haven't noticed any issues with "late" init for the system wall clock either.
Quoted 7 lines
Quoted 5 lines
Or similarly, the SOC family may have a powerful RTC ("arch specific") with alarm and system wakeup capabilities, but it may not be the system's battery backed clock ... so that RTC would be initialized from another one, which is accessed using I2C/SPI/etc. (RTCs integrated on SOCs evidently have a hard time being as power-miserly as the discrete ones.)
SOC?
System-On-Chip ... one chip has not just CPU+cache, but also memory controllers and enough I/O that few external chips are needed. The one in your cell phone is probably Linux-capable; the one in your microwave probably isn't. (For some sophisticated examples, www.omap.com ... click the left menu then look at the chips' block diagrams to see what cell phone CPUs look like.)
Quoted 7 lines
Quoted 4 lines
The approach in this patch doesn't seem to play well with those scenarios, because it expects to do timekeeping setup long before the system's RTC is necessarily going to be available ... and doesn't have an answer for those cases where the RTC is unavailable before e.g.
late_initcall().That's fair. While the patch I just sent didn't consider these cases, I don't think these scenarios break the intent of what I'm wanting to do.
Not in the big picture, nope. Which is why I called it an implementation issue. I've also noticed the mess of arch/platform specific clock setup wierdness, and was glad to see you start fixing it. :)
Quoted 5 lines
Quoted 3 lines
I'd be more interested in something that improves on
CONFIG_RTC_HCTOSYS, and for example addresses the need to update the system wall time from such RTCs after resume, not just at boot time.Agreed.
There's some ARM code to restore the jiffies-vs-rtc clock offset on resume, which seems like a candidate for wider use.
Quoted 8 lines
Basically what I'm trying to provide w/
read_persistent_clock()is:
1) A method for generic timekeeping to access a battery backed clock to avoid potential suspend/resume ordering problems.
2) A generic interface to a low-res hardware driven clock for the hangcheck-timer and other watchdog timers.
3) To cleanup the arch time initialization code and encapsulate xtimeDo you think the RTC interface is sufficient for this?
I think the resume side needs some tweaks, but basically yes. (The resume sequencing, like the suspend sequencing, is excessively dumb. It'd probably be good to resume RTCs before drivers that care about elapsed time, for example ... )
I'm not entirely sure I see RTC as a clocksource, but maybe that'd make sense...
- Dave
Quoted 7 lines · David Brownell
Quoted 6 lines · john stultz
Quoted 4 lines · David Brownell
Hmm, this seems to ignore the RTC framework rather entirely, which seems to me like the wrong implementation approach. You'd likely have noticed that if you had supported a few ARM boards. :)
Quoted 7 lines
Quoted 3 lines
Good point. I've not spent enough time looking at the RTC code, and it looks like it has potential to resolve some of the issues I'm looking at. Although, how common is it really?
Increasingly so; it solves a lot of reinvent-the-wheel problems, much like the recent generic time-of-day updates are doing. And just like with clocksource updates, not all architectures have converted yet!
Is a breakdown somewhere of what is covered at this point? The two letters followed by six number driver files make it difficult to see which arches use what. :)
Quoted 5 lines
Quoted 2 lines
Could we currently replace all
arches xtime initialization code with it?Not yet, I think, mostly because RTCs are still being converted to that framework.
With a list of arches that are covered, I'll start working on patches to do so.
Quoted 3 lines
For ACPI PCs, I converted
drivers/char/rtc.cto that framework (see:
http://article.gmane.org/gmane.linux.kernel/427493) covering most hardware over the last few years.
Cool.
Quoted 5 lines
Quoted 2 lines
I suspect that would require
it to be built at all times. Would that be acceptable?It's just a question of _which_ code to build all the time, right? Having it be standardized core code would seem appropriate.
Yea. We also need to make sure every arch has something that will work w/o special compile time config options.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Here's a fairly common scenario for an embedded system: the battery backed clock is accessed through I2C or SPI, rather than an ISA style direct register access, so it's not accessible until after those driver stacks have initialized.
Quoted 8 lines
Quoted 2 lines
Yea. Although late initialization isn't that much of an issue in my mind.
... except that unless I traced the code wrong, your current patch is initializing the (wall) clock extremely early ... ergo my comment about assuming ISA style access. I'm happy to call that a minor implementation artifact though, since I haven't noticed any issues with "late" init for the system wall clock either.
Indeed. If we cannot initialize xtime early, that's not an issue. settimeofday() will fix that up. However the suspend/resume ordering issue is more difficult. On resume we need to have something usable before we resume the timekeeping code in order to correctly increment xtime and jiffies in one atomic action.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Or similarly, the SOC family may have a powerful RTC ("arch specific") with alarm and system wakeup capabilities, but it may not be the system's battery backed clock ... so that RTC would be initialized from another one, which is accessed using I2C/SPI/etc. (RTCs integrated on SOCs evidently have a hard time being as power-miserly as the discrete ones.)
Quoted 7 lines
Quoted 1 line
SOC?
System-On-Chip ... one chip has not just CPU+cache, but also memory controllers and enough I/O that few external chips are needed. The one in your cell phone is probably Linux-capable; the one in your microwave probably isn't. (For some sophisticated examples, www.omap.com ... click the left menu then look at the chips' block diagrams to see what cell phone CPUs look like.)
Thanks for the clarification.
With regards to more powerful RTCs, those I don't really care too much about. I just want access the battery backed clock for suspend/resume and watchdog uses.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
I'd be more interested in something that improves on
CONFIG_RTC_HCTOSYS, and for example addresses the need to update the system wall time from such RTCs after resume, not just at boot time.Agreed.
There's some ARM code to restore the jiffies-vs-rtc clock offset on resume, which seems like a candidate for wider use.
Where is this code? I'm interested here, but I suspect for correctness that bit should be under the control of the timekeeping core.
Quoted 7 lines
Quoted 7 lines
Basically what I'm trying to provide w/
read_persistent_clock()is:
1) A method for generic timekeeping to access a battery backed clock to avoid potential suspend/resume ordering problems.
2) A generic interface to a low-res hardware driven clock for the hangcheck-timer and other watchdog timers.
3) To cleanup the arch time initialization code and encapsulate xtime
Quoted 6 lines
Quoted 1 line
Do you think the RTC interface is sufficient for this?
I think the resume side needs some tweaks, but basically yes. (The resume sequencing, like the suspend sequencing, is excessively dumb. It'd probably be good to resume RTCs before drivers that care about elapsed time, for example ... )
Ok. I'm not totally sold on it, but I'll start looking in more detail here. Starting w/ i386 and possibly moving across the rest of the arches.
Quoted 2 lines
I'm not entirely sure I see RTC as a clocksource, but maybe that'd make sense...
Ehh. That's definitely not the direction I'm trying to go (although I guess one could possibly do so). I just want the 3 items above. :)
thanks
-john
Quoted 12 lines · john stultz
Quoted 11 lines · David Brownell
Quoted 9 lines · john stultz
Quoted 4 lines · David Brownell
Hmm, this seems to ignore the RTC framework rather entirely, which seems to me like the wrong implementation approach. You'd likely have noticed that if you had supported a few ARM boards. :)
Good point. I've not spent enough time looking at the RTC code, and it looks like it has potential to resolve some of the issues I'm looking at. Although, how common is it really?
Quoted 7 lines
Quoted 3 lines
Increasingly so; it solves a lot of reinvent-the-wheel problems, much like the recent generic time-of-day updates are doing. And just like with clocksource updates, not all architectures have converted yet!
Is a breakdown somewhere of what is covered at this point? The two letters followed by six number driver files make it difficult to see which arches use what. :)
A lot of those are part numbers for discrete chips, and some names are for various ARM SOC families.
Seems to me the better question is what non-framework RTC drivers are still in the tree ... and actually a good first pass at answering that is that every platform touched in your original patch was NOT using the RTC class framework. (Not the answer you wanted, right?) And drivers/char/*rtc* users. There may be a few other RTCs elsewhere in the tree.
It seems all in-kernel ARM platforms except ARM Ltd's own "Integrator" use the new framework instead of the older arch/arm/common/rtctime.c code (from which the RTC framework evolved). And I know there are other RTC drivers and patches in the queue. ARMs that haven't yet merged to mainline will, as usual, pay the price for their perfidy.
Quoted 4 lines
However the suspend/resume ordering
issue is more difficult. On resume we need to have something usable before we resume the timekeeping code in order to correctly increment xtime and jiffies in one atomic action.
Right; not something addressed in your original patch, or by many current drivers AFAIK. For the longest time, swsusp resume got that wrong...
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
I'd be more interested in something that improves on
CONFIG_RTC_HCTOSYS, and for example addresses the need to update the system wall time from such RTCs after resume, not just at boot time.Agreed.
There's some ARM code to restore the jiffies-vs-rtc clock offset on resume, which seems like a candidate for wider use.
Where is this code? I'm interested here, but I suspect for correctness that bit should be under the control of the timekeeping core.
{save,restore}_time_delta() in
include/asm-arm/mach/time.h
arch/arm/kernel/time.c
The rtc-at91.c code is layered classically ... RTC is the first platform device registered, and thus resumed, but it's resumed after the jiffies timer sysdev (thus with IRQs enabled). So it will restore the time delta very early in the resume sequence. (PXA uses a different approach.)
Presumably this is stuff that should be done by the RTC class resume() method, probably for the CONFIG_RTC_HCTOSYS_DEVICE clock (though there could be a better RTC ... one that's being NTP-corrected). That'd be no sooner than 2.6.19, which adds new class-level suspend()/resume() calls to help offload individual drivers.
Quoted 5 lines
Quoted 2 lines
I'm not entirely sure I see RTC as a clocksource, but maybe that'd make sense...
Ehh. That's definitely not the direction I'm trying to go (although I guess one could possibly do so). I just want the 3 items above. :)
Glad to hear it! Though strictly speaking it _ought_ to make sense to declare RTCs as clock sources. They are clocks, after all. I can even imagine systems where the RTC issues (non-dynamic) tick interupts; some RTCs (like those on PCs) are happy giving e.g 256 or 1024 irq/sec.
- Dave
Quoted 21 lines · David Brownell
Quoted 12 lines · john stultz
Quoted 11 lines · David Brownell
Quoted 9 lines · john stultz
Quoted 4 lines · David Brownell
Hmm, this seems to ignore the RTC framework rather entirely, which seems to me like the wrong implementation approach. You'd likely have noticed that if you had supported a few ARM boards. :)
Good point. I've not spent enough time looking at the RTC code, and it looks like it has potential to resolve some of the issues I'm looking at. Although, how common is it really?
Quoted 7 lines
Quoted 3 lines
Increasingly so; it solves a lot of reinvent-the-wheel problems, much like the recent generic time-of-day updates are doing. And just like with clocksource updates, not all architectures have converted yet!
Is a breakdown somewhere of what is covered at this point? The two letters followed by six number driver files make it difficult to see which arches use what. :)
Quoted 8 lines
A lot of those are part numbers for discrete chips, and some names are for various ARM SOC families.
Seems to me the better question is what non-framework RTC drivers are still in the tree ... and actually a good first pass at answering that is that every platform touched in your original patch was NOT using the RTC class framework. (Not the answer you wanted, right?) And
drivers/char/*rtc*users. There may be a few other RTCs elsewhere in the tree.
Uh, so you're saying the RTC framework is basically ARM only? Eek. You're right, that's not the answer I wanted to hear!
Quoted 7 lines
It seems all in-kernel ARM platforms except ARM Ltd's own "Integrator" use the new framework instead of the older
arch/arm/common/rtctime.ccode (from which the RTC framework evolved). And I know there are other RTC drivers and patches in the queue. ARMs that haven't yet merged to mainline will, as usual, pay the price for their perfidy.
Quoted 7 lines
Quoted 4 lines
However the suspend/resume ordering
issue is more difficult. On resume we need to have something usable before we resume the timekeeping code in order to correctly increment xtime and jiffies in one atomic action.Right; not something addressed in your original patch, or by many current drivers AFAIK. For the longest time, swsusp resume got that wrong...
True, I left it out of my patch since the generic suspend/resume takes a bit more arch code then I had. You can find my current arch generic portion of the patch (including suspend/resume adjustments) below. Following this one would be the i386 patch that provides read_persistent_clock() and removes all i386 arch specific timekeeping adjustments (I can send it on request if needed).
Quoted 14 lines
Quoted 11 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
I'd be more interested in something that improves on
CONFIG_RTC_HCTOSYS, and for example addresses the need to update the system wall time from such RTCs after resume, not just at boot time.Agreed.
There's some ARM code to restore the jiffies-vs-rtc clock offset on resume, which seems like a candidate for wider use.
Where is this code? I'm interested here, but I suspect for correctness that bit should be under the control of the timekeeping core.
{save,restore}_time_delta() in
Quoted 7 lines
include/asm-arm/mach/time.h
arch/arm/kernel/time.cThe
rtc-at91.ccode is layered classically ... RTC is the first platform device registered, and thus resumed, but it's resumed after the jiffies timer sysdev (thus with IRQs enabled). So it will restore the time delta very early in the resume sequence. (PXA uses a different approach.)
Hmmm. So looking at the code, ARM doesn't update jiffies on resume?
Quoted 5 lines
Presumably this is stuff that should be done by the RTC class
resume()method, probably for theCONFIG_RTC_HCTOSYS_DEVICEclock (though there could be a better RTC ... one that's being NTP-corrected). That'd be no sooner than 2.6.19, which adds new class-level suspend()/resume() calls to help offload individual drivers.
Hrm. I'm a bit skeptical that the RTC resume code should update the timekeeping code instead of the timekeeping code doing it. It seems that it would cause additional complexity (what if there are two RTC devices?) and would still have some of the suspend/resume ordering issues I'm worried about.
thanks
-john
include/linux/time.h | 1 + kernel/timer.c | 36 ++++++++++++++++++++++++++++++------ 2 files changed, 31 insertions(+), 6 deletions(-)
include/linux/time.h +1 -0
@@ -92,6 +92,7 @@ extern struct timespec xtime;extern struct timespec wall_to_monotonic;extern seqlock_t xtime_lock; +extern unsigned long read_persistent_clock (void );void timekeeping_init (void );static inline unsigned long get_seconds (void )
kernel/timer.c +30 -6
@@ -47,6 +47,9 @@ static void time_interpolator_update(lon#define time_interpolator_update (x)#endif +/* jiffies at the most recent update of wall time */ +unsigned long wall_jiffies =INITIAL_JIFFIES ; + u64 jiffies_64 __cacheline_aligned_in_smp = INITIAL_JIFFIES;EXPORT_SYMBOL (jiffies_64);
@@ -953,6 +956,14 @@ int timekeeping_is_continuous(void)return ret; } +/* Weak dummy function for arches that do not yet support it. +* XXX - Do be sure to remove it once all arches implement it. +*/ +unsigned long __attribute__((weak))read_persistent_clock (void ) +{ +return 0 ; +} +/* * timekeeping_init - Initializes the clocksource and common timekeeping values */
@@ -961,15 +972,22 @@ void __init timekeeping_init(void)unsigned long flags;write_seqlock_irqsave (&xtime_lock, flags); +ntp_clear (); + clock =clocksource_get_next ();clocksource_calculate_interval (clock, tick_nsec); clock->cycle_last =clocksource_read (clock); -ntp_clear (); + + xtime.tv_sec =read_persistent_clock (); + xtime.tv_nsec = (INITIAL_JIFFIES %HZ ) * (NSEC_PER_SEC /HZ ); +set_normalized_timespec (&wall_to_monotonic, + -xtime.tv_sec, -xtime.tv_nsec);write_sequnlock_irqrestore (&xtime_lock, flags); }static int timekeeping_suspended; +static unsigned long timekeeping_suspend_time;/* * timekeeping_resume - Resumes the generic timekeeping subsystem. * @dev: unused
@@ -980,10 +998,18 @@ static int timekeeping_suspended;*/ static int timekeeping_resume (struct sys_device *dev) { -unsigned long flags; +unsigned long flags, now;write_seqlock_irqsave (&xtime_lock, flags); -/* restart the last cycle value */ +/* calculate time suspended using the persistent clock */ + now =read_persistent_clock (); +if (now && (now > timekeeping_suspend_time)) { +unsigned long sleep_length = now - timekeeping_suspend_time; + xtime.tv_sec += sleep_length; + jiffies_64 += sleep_length *HZ ; + wall_jiffies += sleep_length *HZ ; + } +/* re-base the last cycle value */ clock->cycle_last =clocksource_read (clock); clock->error =0 ; timekeeping_suspended =0 ;
@@ -997,6 +1023,7 @@ static int timekeeping_suspend(struct sywrite_seqlock_irqsave (&xtime_lock, flags); timekeeping_suspended =1 ; + timekeeping_suspend_time =read_persistent_clock ();write_sequnlock_irqrestore (&xtime_lock, flags);return 0 ; }
@@ -1227,9 +1254,6 @@ static inline void calc_load(unsigned lo } } -/* jiffies at the most recent update of wall time */ -unsigned long wall_jiffies =INITIAL_JIFFIES ; -/* * This read-write spinlock protects us from races in SMP while * playing with xtime and avenrun.
Quoted 8 lines · john stultz
Quoted 5 lines
Seems to me the better question is what non-framework RTC drivers are still in the tree ... and actually a good first pass at answering that is that every platform touched in your original patch was NOT using the RTC class framework. (Not the answer you wanted, right?) And
drivers/char/*rtc*users. There may be a few other RTCs elsewhere in the tree.Uh, so you're saying the RTC framework is basically ARM only? Eek. You're right, that's not the answer I wanted to hear!
That's not what I said, no ... I hope you don't think that only ARM chips can use I2C or SPI to connect to battery backed RTCs!!
Maybe you should think of it more like "so far mostly embedded systems use that framework" ... since it's embedded systems that need to cope with that variety in hardware. RTCs are not architectural features, but are just implementation choices made during board or SOC design. Since custom designs are endemic to embedded hardware, that's where the need for this RTC framework is critical. A huge chunk of those markets use ARM (it's the most popular embedded 32bit CPU), but I know various MIPS and PPC boards also rely on those Linux RTC framework drivers.
The RTC framework is no more ARM-only than the generic TOD framework is x86-only. But those changes did start from different corners of the Linux market, which likely explains some of the surprise associated with this little collision. (If rtc-acpi got a bit more attention, that'd surely help raise awareness outside the embedded space...)
Quoted 11 lines
Quoted 9 lines
{save,restore}_time_delta() in
include/asm-arm/mach/time.h
arch/arm/kernel/time.cThe
rtc-at91.ccode is layered classically ... RTC is the first platform device registered, and thus resumed, but it's resumed after the jiffies timer sysdev (thus with IRQs enabled). So it will restore the time delta very early in the resume sequence. (PXA uses a different approach.)Hmmm. So looking at the code, ARM doesn't update jiffies on resume?
Now that you mention it ... that's right. Just the wall clocks. If you want to argue that's buglike, take it up with Russell King. Presumably some other architectures do advance jiffies, not just the wall clock?
Quoted 11 lines
Quoted 5 lines
Presumably this is stuff that should be done by the RTC class
resume()method, probably for theCONFIG_RTC_HCTOSYS_DEVICEclock (though there could be a better RTC ... one that's being NTP-corrected). That'd be no sooner than 2.6.19, which adds new class-level suspend()/resume() calls to help offload individual drivers.Hrm. I'm a bit skeptical that the RTC resume code should update the timekeeping code instead of the timekeeping code doing it. It seems that it would cause additional complexity (what if there are two RTC devices?) and would still have some of the suspend/resume ordering issues I'm worried about.
Could be. "Two RTCs" scenarios are real, as I've mentioned before. I'll leave it to you to sort this stuff out, you seem to have a good handle on it all ... I was mostly concerned that you incorporate the RTC class support into your updates. Neither of us created the mess surrounding persistent wall clocks, but if you want to unify things, then you should build on the existing RTC framework or else come up with a better one ...
- Dave
Quoted 14 lines · David Brownell
Quoted 8 lines · john stultz
Quoted 5 lines
Seems to me the better question is what non-framework RTC drivers are still in the tree ... and actually a good first pass at answering that is that every platform touched in your original patch was NOT using the RTC class framework. (Not the answer you wanted, right?) And
drivers/char/*rtc*users. There may be a few other RTCs elsewhere in the tree.Uh, so you're saying the RTC framework is basically ARM only? Eek. You're right, that's not the answer I wanted to hear!
That's not what I said, no ... I hope you don't think that only ARM chips can use I2C or SPI to connect to battery backed RTCs!!
Quoted 8 lines
Maybe you should think of it more like "so far mostly embedded systems use that framework" ... since it's embedded systems that need to cope with that variety in hardware. RTCs are not architectural features, but are just implementation choices made during board or SOC design. Since custom designs are endemic to embedded hardware, that's where the need for this RTC framework is critical. A huge chunk of those markets use ARM (it's the most popular embedded 32bit CPU), but I know various MIPS and PPC boards also rely on those Linux RTC framework drivers.
No, I'm sorry, I realize the RTC interface has the potential to be more widely used. I'm just a bit frustrated that in order to utilize the RTC framework for what I'm trying to do, I have to first implement RTC drivers for 90% of the arches. :(
Also the "RTC might not be available when you need it" issue makes it uh.. difficult to use. :)
So if we go w/ the "it may not be available, so always assume it isn't" way of thinking, it forces us to rely upon the RTC driver(s) to resume time (which means every RTC, no matter how simple has to have suspend/resume hooks and call settimeofday at least).
I don't really like that uses-graph (I can imagine someone system not resuming properly because they forgot to compile in the right RTC driver). Also it doesn't resolve the timekeeing resume ordering issue I'm trying to address.
But we might have to deal with it. Just to make sure we can balance this properly, what is the percentage of RTC drivers where the might not be available at timekeeping_resume()? If it is small, it might be reasonable to special case them (and by "special case", i *don't* mean ignore :)
Quoted 5 lines
The RTC framework is no more ARM-only than the generic TOD framework is x86-only. But those changes did start from different corners of the Linux market, which likely explains some of the surprise associated with this little collision. (If rtc-acpi got a bit more attention, that'd surely help raise awareness outside the embedded space...)
Looking at the rtc-acpi code, it describes itself as being AT compatible (ie: The old CMOS clock), but its not clear if it requires ACPI or not to work. Further, does it work on x86_64, or ia64 as well?
Another comment, drivers/rtc/ is a bit overwhelming. Same with the Kconfig. Is there any way it could be broken up so arch-specific RTCs aren't shown on arches that do not support them? Or maybe there should be two classes, the arch-defined RTCs and then the generic ones?
Your thoughts?
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
{save,restore}_time_delta() in
include/asm-arm/mach/time.h
arch/arm/kernel/time.cThe
rtc-at91.ccode is layered classically ... RTC is the first platform device registered, and thus resumed, but it's resumed after the jiffies timer sysdev (thus with IRQs enabled). So it will restore the time delta very early in the resume sequence. (PXA uses a different approach.)
Quoted 5 lines
Quoted 1 line
Hmmm. So looking at the code, ARM doesn't update jiffies on resume?
Now that you mention it ... that's right. Just the wall clocks. If you want to argue that's buglike, take it up with Russell King. Presumably some other architectures do advance jiffies, not just the wall clock?
Well, its a "some arches do, some arches don't" discrepancy, rather then a specific bug. It was my hope that by putting the logic in generic code we would get consistent behavior across arches.
Quoted 12 lines
Quoted 11 lines
Quoted 5 lines
Presumably this is stuff that should be done by the RTC class
resume()method, probably for theCONFIG_RTC_HCTOSYS_DEVICEclock (though there could be a better RTC ... one that's being NTP-corrected). That'd be no sooner than 2.6.19, which adds new class-level suspend()/resume() calls to help offload individual drivers.Hrm. I'm a bit skeptical that the RTC resume code should update the timekeeping code instead of the timekeeping code doing it. It seems that it would cause additional complexity (what if there are two RTC devices?) and would still have some of the suspend/resume ordering issues I'm worried about.
Quoted 6 lines
Could be. "Two RTCs" scenarios are real, as I've mentioned before. I'll leave it to you to sort this stuff out, you seem to have a good handle on it all ... I was mostly concerned that you incorporate the RTC class support into your updates. Neither of us created the mess surrounding persistent wall clocks, but if you want to unify things, then you should build on the existing RTC framework or else come up with a better one ...
I do appreciate you bringing these point up. I do think the RTC class looks promising, but until its widely implemented, its difficult to use/rely upon generically.
A thought: The approach taken w/ the generic timekeeping was that by using the jiffies clocksource, we could stagger the transition. So the clocksource code can be used generically, because all arches are guaranteed to have atleast one clocksource.
Is there some way we can take a similar approach, by combining the read_persistent_clock() and the RTC framework? Basically creating a dummy RTC that would be on every arch, implementing only the read method which would call something like read_persistent_clock(). This way the RTC infrastructure could be used on all arches, while the conversion was still happening.
Alternatively it may be that the real solution is to either introduce resume ordering levels or explicitly call timekeeping resume, instead of leaving it to the sysclass logic.
thanks
-john
Quoted 4 lines · john stultz
No, I'm sorry, I realize the RTC interface has the potential to be more widely used. I'm just a bit frustrated that in order to utilize the RTC framework for what I'm trying to do, I have to first implement RTC drivers for 90% of the arches. :(
Maybe not ... you could do a lot with just a hook function. Your original patch provided one hook (an arch-level function call) assuming the RTC is always accessible with IRQs blocked. The RTC framework could provide such a similar hook too ... best done through a function pointer, though. It shouldn't be a kernel build error if there's no RTC; userspace can use an external one via NTP, load a module later than you'd like, etc.
Then be sure to call that hook from some can-sleep context, and you're as set for the boot/init issue as possible. (That is, no luck if the RTC is in a module loaded after init starts.)
Quoted 2 lines
Also the "RTC might not be available when you need it" issue makes it uh.. difficult to use. :)
I can't see that "when you need it" thing. Especially since you had proposed that "late" wall clock init was not a problem ...
Quoted 4 lines
So if we go w/ the "it may not be available, so always assume it isn't" way of thinking, it forces us to rely upon the RTC driver(s) to resume time (which means every RTC, no matter how simple has to have suspend/resume hooks and call settimeofday at least).
No, that was the point of my comment about using the new class level suspend/resume calls. The RTC drivers wouldn't be responsible for that; the RTC framework would be. RTC drivers may still want the suspend/resume hooks to make sure they issue system wakeup events, and so on, but no longer for maintaining the wall clock. I'll send a patch (of the "it compiles" type) later.
Quoted 3 lines
I don't really like that uses-graph (I can imagine someone system not resuming properly because they forgot to compile in the right RTC driver).
Without the right RTC support, they'd not be getting initial clock setup right either ... same difference, same userspace fix (using external RTC, via NTP etc).
Quoted 2 lines
Also it doesn't resolve the timekeeping resume ordering issue I'm trying to address.
Worth exploring that a bit more. Exactly what is the issue? I'm not sure your first explanation came across in enough depth...
Quoted 5 lines
But we might have to deal with it. Just to make sure we can balance this properly, what is the percentage of RTC drivers where the might not be available at
timekeeping_resume()? If it is small, it might be reasonable to special case them (and by "special case", i *don't* mean ignore :)
Basically 100%. It's because timekeeping_resume() applies to a (fake) sys_device, and the sys_device resume phase kicks in before "real" drivers resume, and with IRQs blocked ... ergo before sleeping calls can be issued (e.g. waiting for I2C or SPI access to complete).
The RTC drivers are all "real" device drivers, and the RTC framework itself issues sleeping calls, like mutex_lock.
Quoted 9 lines
Quoted 5 lines
The RTC framework is no more ARM-only than the generic TOD framework is x86-only. But those changes did start from different corners of the Linux market, which likely explains some of the surprise associated with this little collision. (If rtc-acpi got a bit more attention, that'd surely help raise awareness outside the embedded space...)
Looking at the rtc-acpi code, it describes itself as being AT compatible (ie: The old CMOS clock), but its not clear if it requires ACPI or not to work. Further, does it work on
x86_64, or ia64 as well?
It's called "rtc-acpi" since it requires ACPI ... in fact, PNPACPI is what provides the driver model device it binds to. Plus it hooks into ACPI to get the wakeup function, and the extra register options. I'm running it on both i386 and x86_64. So like I said, most modern PCs should be able to run it just fine. IA64 uses ACPI, so presumably it should be able to use the driver too.
The core of the driver could be reused on some non-PC platforms, some not-modern PCs, and on ACPI platforms without PNPACPI ... given the addition of platform code to register a platform_device. And for the ACPI-but-not-PNPCACPI configs, wakeup and the extra registers could still be used.
Someone would have to provide the relevant patches; I'd not mind if the driver were renamed to e.g. "rtc-cmos" at that point.
Quoted 3 lines
Another comment,
drivers/rtc/is a bit overwhelming. Same with theKconfig. Is there any way it could be broken up so arch-specific RTCs aren't shown on arches that do not support them?
They aren't. You won't see the various SOC's RTC unless configuring a kernel for that SOC ...
Quoted 4 lines
Or maybe there should
be two classes, the arch-defined RTCs and then the generic ones?Your thoughts?
I CC'd Allesandro Zummo, maintaining the RTC framework ... I'm sure he's noticed the explosion of drivers! It's not so overwhelming as drivers/net though... :)
Sectioning Kconfig seems to me like a reasonable and simple change, first the integrated/arch RTCs (link them first too, so they'll be "rtc0" when not modular) then the discrete ones ... maybe the I2C ones first (in alphabetical order), then SPI ones, memory mapped ones, etc. Any janitors reading this thread?
But I don't see a need for multiple classes; they're still all just RTCs, the type of implementation isn't supposed to matter at all.
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 9 lines
{save,restore}_time_delta() in
include/asm-arm/mach/time.h
arch/arm/kernel/time.cThe
rtc-at91.ccode is layered classically ... RTC is the first platform device registered, and thus resumed, but it's resumed after the jiffies timer sysdev (thus with IRQs enabled). So it will restore the time delta very early in the resume sequence. (PXA uses a different approach.)Hmmm. So looking at the code, ARM doesn't update jiffies on resume?
Quoted 7 lines
Quoted 3 lines
Now that you mention it ... that's right. Just the wall clocks. If you want to argue that's buglike, take it up with Russell King. Presumably some other architectures do advance jiffies, not just the wall clock?
Well, its a "some arches do, some arches don't" discrepancy, rather then a specific bug. It was my hope that by putting the logic in generic code we would get consistent behavior across arches.
I'm all for that!
Quoted 19 lines
Quoted 12 lines
Quoted 11 lines
Quoted 5 lines
Presumably this is stuff that should be done by the RTC class
resume()method, probably for theCONFIG_RTC_HCTOSYS_DEVICEclock (though there could be a better RTC ... one that's being NTP-corrected). That'd be no sooner than 2.6.19, which adds new class-level suspend()/resume() calls to help offload individual drivers.Hrm. I'm a bit skeptical that the RTC resume code should update the timekeeping code instead of the timekeeping code doing it. It seems that it would cause additional complexity (what if there are two RTC devices?) and would still have some of the suspend/resume ordering issues I'm worried about.
Quoted 6 lines
Could be. "Two RTCs" scenarios are real, as I've mentioned before. I'll leave it to you to sort this stuff out, you seem to have a good handle on it all ... I was mostly concerned that you incorporate the RTC class support into your updates. Neither of us created the mess surrounding persistent wall clocks, but if you want to unify things, then you should build on the existing RTC framework or else come up with a better one ...
Quoted 3 lines
I do appreciate you bringing these point up. I do think the RTC class looks promising, but until its widely implemented, its difficult to use/rely upon generically.
Every Linux system I work on today -- x86, x86_64, multiple ARMs -- supports the RTC framework though. To me it's *already* "widely implemented" ... :)
Quoted 11 lines
A thought: The approach taken w/ the generic timekeeping was that by using the jiffies clocksource, we could stagger the transition. So the clocksource code can be used generically, because all arches are guaranteed to have atleast one clocksource.
Is there some way we can take a similar approach, by combining the
read_persistent_clock()and the RTC framework? Basically creating a dummy RTC that would be on every arch, implementing only the read method which would call something likeread_persistent_clock(). This way the RTC infrastructure could be used on all arches, while the conversion was still happening.
What I sketched above is somewhat similar ... there couldn't be an RTC framework driver, since the platform would need to provide a driver model device in order to use the RTC framework. And as I've already noted, "every arch" can't implement such a method...
But it would be easy for read_persistent_clock() to be a pointer to a function that's filled in by whatever -- non-RTC arch code, or the RTC framework -- and called some appropriate can-sleep place.
Quoted 3 lines
Alternatively it may be that the real solution is to either introduce resume ordering levels or explicitly call timekeeping resume, instead of leaving it to the sysclass logic.
The fact that timekeeping_resume ignores its parameter is IMNSHO a good hint that it shouldn't be shoehorned into the driver model in that way. Now, the RTC framework _does_ fit well into the driver model ... :)
- Dave
Quoted 11 lines · David Brownell
Quoted 4 lines
So if we go w/ the "it may not be available, so always assume it isn't" way of thinking, it forces us to rely upon the RTC driver(s) to resume time (which means every RTC, no matter how simple has to have suspend/resume hooks and call settimeofday at least).
No, that was the point of my comment about using the new class level suspend/resume calls. The RTC drivers wouldn't be responsible for that; the RTC framework would be. RTC drivers may still want the suspend/resume hooks to make sure they issue system wakeup events, and so on, but no longer for maintaining the wall clock. I'll send a patch (of the "it compiles" type) later.
Welcome to "later"!
------------------------
Preliminary RTC class suspend/resume support:
- Inlining the same code used by ARM, save and restore the delta between a selected RTC and the current system time.
- Removes calls to that ARM code from the AT91 and S3C RTCs; the AT91 calls are left as stubs, because a pending patch still needs those (currently empty) routines to properly issue system wakeup events.
- Selects rtc0 by default, else CONFIG_RTC_HCTOSYS_DEVICE. We expect all RTCs to be powered during "real" sleep states, but swsusp/hibernation requires something that's also battery-backed.
Depends on new class suspend/resume methods, added by a patch in the MM tree.
(Preliminary because untested, and because of a FIXME ... nothing recovers from unregistering the selected RTC; that could happen only with badly written RTC drivers though.)
drivers/rtc/class.c +110 -0
@@ -29,6 +29,86 @@ static void rtc_device_release(struct clkfree (rtc); } +#ifdef CONFIG_PM + +/* +* Re-initialize wall clock on resume, to match the delta (calculated +* during suspend) between that and an appropriate RTC. +* +* We assume any RTC is good enough, meaning it stays running during +* system sleep states; but prefer the CONFIG_RTC_HCTOSYS_DEVICE on +* the grounds that it's expected to be battery-backed and thus will +* stay running even during the "off" states used by swsusp. +* +* REVISIT ... we should probably have a way to tell if a given RTC +* will stay powered during the target system state, rather than just +* assume a "valid" configuration (where no RTC that powers off will +* be selected, and no selected RTC will be removed). That could let +* us get rid of the need for CONFIG_RTC_HCTOSYS_DEVICE too... +*/ + +static struct class_device *sleep_rtc; +static struct timespec delta; + +static int rtc_suspend (struct device *dev,pm_message_t mesg) +{ +struct rtc_time tm; +struct timespec time; + +if (!sleep_rtc || dev != sleep_rtc->dev) +return 0 ; + + time.tv_nsec =0 ; + +rtc_read_time (sleep_rtc, &tm); +rtc_tm_to_time (&tm, &time.tv_sec); + +set_normalized_timespec (&delta, + xtime.tv_sec - time.tv_sec, + xtime.tv_nsec - time.tv_nsec); + +pr_debug ("%s: %s %4d-%02d-%02d %02d:%02d:%02d\n" , + sleep_rtc->class_id,"suspend" , +1900 + tm.tm_year, tm.tm_mon, tm.tm_mday, + tm.tm_hour, tm.tm_min, tm.tm_sec); + +return 0 ; +} + +static int rtc_resume (struct device *dev) +{ +struct rtc_time tm; +struct timespec time; + +/* REVISIT some swsusp configurations may feed us an RTC that +* lost power ... we should probably try detecting that, and +* refuse to update the time using a bogus clock. +*/ + +if (!sleep_rtc || dev != sleep_rtc->dev) +return 0 ; + + time.tv_nsec =0 ; + +rtc_read_time (sleep_rtc, &tm); +rtc_tm_to_time (&tm, &time.tv_sec); + +set_normalized_timespec (&time, + xtime.tv_sec - time.tv_sec, + xtime.tv_nsec - time.tv_nsec); +do_settimeofday (&time); + +pr_debug ("%s: %s %4d-%02d-%02d %02d:%02d:%02d\n" , + sleep_rtc->class_id,"resume" , +1900 + tm.tm_year, tm.tm_mon, tm.tm_mday, + tm.tm_hour, tm.tm_min, tm.tm_sec); + +return 0 ; +} + +#endif + +/** * rtc_device_register - register w/ RTC class * @dev: the device to register
@@ -85,6 +165,21 @@ struct rtc_device *rtc_device_register(cif (err) goto exit_kfree; +#ifdef CONFIG_PM +mutex_lock (&idr_lock); +#ifdef CONFIG_RTC_HCTOSYS_DEVICE +if (sleep_rtc &&strncmp (rtc->class_dev.class_id, +CONFIG_RTC_HCTOSYS_DEVICE , +BUS_ID_SIZE ) ==0 ) { +rtc_class_close (sleep_rtc); + sleep_rtc =NULL ; + } +#endif +if (!sleep_rtc) + sleep_rtc =rtc_class_open (rtc->class_dev.class_id); +mutex_unlock (&idr_lock); +#endif +dev_info (dev,"rtc core: registered %s as %s\n" , rtc->name, rtc->class_dev.class_id);
@@ -113,6 +208,17 @@ EXPORT_SYMBOL_GPL(rtc_device_register);*/ void rtc_device_unregister (struct rtc_device *rtc) { +#ifdef CONFIG_PM +mutex_lock (&idr_lock); +if (&rtc->class_dev == sleep_rtc) { +rtc_class_close (sleep_rtc); + sleep_rtc =NULL ; +/* FIXME */ +printk (KERN_WARNING "rtc: now what to use during sleep??\n" ); + } +mutex_unlock (&idr_lock); +#endif +mutex_lock (&rtc->ops_lock); rtc->ops =NULL ;mutex_unlock (&rtc->ops_lock);
@@ -134,6 +240,10 @@ static int __init rtc_init(void)printk (KERN_ERR "%s: couldn't create class\n" , __FILE__);return PTR_ERR (rtc_class); } +#ifdef CONFIG_PM + rtc_class->suspend = rtc_suspend; + rtc_class->resume = rtc_resume; +#endif return 0 ; }
drivers/rtc/rtc-at91.c +2 -27
@@ -339,38 +339,13 @@ static struct timespec at91_rtc_delta;static int at91_rtc_suspend (struct platform_device *pdev,pm_message_t state) { -struct rtc_time tm; -struct timespec time; - - time.tv_nsec =0 ; - -/* calculate time delta for suspend */ -at91_rtc_readtime (&pdev->dev, &tm); -rtc_tm_to_time (&tm, &time.tv_sec); -save_time_delta (&at91_rtc_delta, &time); - -pr_debug ("%s(): %4d-%02d-%02d %02d:%02d:%02d\n" , __FUNCTION__, -1900 + tm.tm_year, tm.tm_mon, tm.tm_mday, - tm.tm_hour, tm.tm_min, tm.tm_sec); - +/* STUB, pending merge of rtc wakeup patch */ return 0 ; }static int at91_rtc_resume (struct platform_device *pdev) { -struct rtc_time tm; -struct timespec time; - - time.tv_nsec =0 ; - -at91_rtc_readtime (&pdev->dev, &tm); -rtc_tm_to_time (&tm, &time.tv_sec); -restore_time_delta (&at91_rtc_delta, &time); - -pr_debug ("%s(): %4d-%02d-%02d %02d:%02d:%02d\n" , __FUNCTION__, -1900 + tm.tm_year, tm.tm_mon, tm.tm_mday, - tm.tm_hour, tm.tm_min, tm.tm_sec); - +/* STUB, pending merge of rtc wakeup patch */ return 0 ; }#else
drivers/rtc/rtc-s3c.c +0 -22
@@ -536,37 +536,15 @@ static int ticnt_save;static int s3c_rtc_suspend (struct platform_device *pdev,pm_message_t state) { -struct rtc_time tm; -struct timespec time; - - time.tv_nsec =0 ; -/* save TICNT for anyone using periodic interrupts */ - ticnt_save =readb (S3C2410_TICNT ); - -/* calculate time delta for suspend */ - -s3c_rtc_gettime (&pdev->dev, &tm); -rtc_tm_to_time (&tm, &time.tv_sec); -save_time_delta (&s3c_rtc_delta, &time);s3c_rtc_enable (pdev,0 ); -return 0 ; }static int s3c_rtc_resume (struct platform_device *pdev) { -struct rtc_time tm; -struct timespec time; - - time.tv_nsec =0 ; -s3c_rtc_enable (pdev,1 ); -s3c_rtc_gettime (&pdev->dev, &tm); -rtc_tm_to_time (&tm, &time.tv_sec); -restore_time_delta (&s3c_rtc_delta, &time); -writeb (ticnt_save,S3C2410_TICNT );return 0 ; }
By the way, one furher observation:
Quoted 8 lines
Quoted 5 lines
Presumably this is stuff that should be done by the RTC class
resume()method, probably for theCONFIG_RTC_HCTOSYS_DEVICEclock (though there could be a better RTC ... one that's being NTP-corrected). That'd be no sooner than 2.6.19, which adds new class-level suspend()/resume() calls to help offload individual drivers.Hrm. I'm a bit skeptical that the RTC resume code should update the timekeeping code instead of the timekeeping code doing it.
But how could timekeeping code know when the RTC is accessible again?
It might be unusable until after e.g. an I2C controller and then the RTC are resumed (and maybe a parent to the I2C controller) ... the class resume mechanism piggybacks on the normal resume sequencing, so doing it that way would automatically resolve that issue.
Quoted 4 lines
It seems that
it would cause additional complexity (what if there are two RTC devices?) and would still have some of the suspend/resume ordering issues I'm worried about.
The RTC framework currently "knows" which RTC is used for the hctosys update, so that approach can be addressed by saving a single pointer and adding some class suspend()/resume() code to test it. No complexity necessary; if it's good enough to set the initial system time, it's good enough to set it again after resume. (And maybe all the RTCs should be getting NTP updates, not just one of them ...)
I think that if you keep in mind that scenario -- I2C used to access the RTC clock -- and make sure it works, you'll notice a few details of your current approach that need to be changed.
In particular, it is impossible for any sys_device's driver resume() to access I2C (since that resume is done with irqs disabled, while i2c controllers normally require irqs to run); but that's what the current version of timekeeping_resume() in your patch would be doing.
- Dave