Skip to contentSkip to conversation

Couple of timekeeping fixes

RFC PATCH In review John Stultz 2 participants 9 messages Dec 10, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Couple of timekeeping fixes

I was looking into the lockdep splat reported by Sasha yesterday and came across a few issues (unfortunately not related) in the timekeeping code.

The first two are issues related to not updating the shadow timekeeper after making changes to the timekeeper structure. This means those updates could be lost the next time we do update_wall_time(), since update_wall_time assumes the shadow_timekeeper is current as well.

The last change is an obvious issue that I should have caught in review, but where we handle notifying the pvclock code if time was set, there's one case in logarithmic_accumulation where we just don't pass that flag down.

Of the three patches, the first is really the most critical. I'm thinking of pushing that one into 3.13, and immediately back to 3.12-stable and 3.10-stable. Then leaving the last two for 3.14, and pushing back to 3.13/10-stable once those changes are merged.

I'm still running some tests on these, but I wanted to send them out as RFCs to get some extra review and thoughts before I send them out for real.

thanks
-john

Cc Thomas Gleixner <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Richard Cochran <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc David Vrabel <[email protected]>

John Stultz (3):
timekeeping: Fix lost updates to tai adjustment
timekeeping: Fix missing timekeeping_update in suspend path
timekeeping: Fix potential lost pv notification of time change

 kernel/time/timekeeping.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
Signature · 1 line
1.8.3.2
#1 lore raw

RFC PATCH 1/3 timekeeping: Fix lost updates to tai adjustment

Since 48cdc135d4840 (Implement a shadow timekeeper), we have to call timekeeping_update() after any adjustment to the timekeeping structure in order to make sure that any adjustments to the structure persist.

Unfortunately, the updates to the tai offset via adjtimex do not trigger this update, causing adjustments to the tai offset to be made and then over-written by the previous value at the next update_wall_time() call.

This patch resovles the issue by calling timekeeping_update() right after setting the tai offset.

Cc Thomas Gleixner <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Richard Cochran <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc stable <[email protected]> #3.10+
---
 kernel/time/timekeeping.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/time/timekeeping.c +2 -1
index 87b4f00..6bad3d9 100644
@@ -610,6 +610,7 @@ void timekeeping_set_tai_offset(s32 tai_offset)
 	raw_spin_lock_irqsave(&timekeeper_lock, flags);
 	write_seqcount_begin(&timekeeper_seq);
 	__timekeeping_set_tai_offset(tk, tai_offset);
+	timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
 	write_seqcount_end(&timekeeper_seq);
 	raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
 	clock_was_set();
@@ -1698,7 +1699,7 @@ int do_adjtimex(struct timex *txc)
 
 	if (tai != orig_tai) {
 		__timekeeping_set_tai_offset(tk, tai);
-		update_pvclock_gtod(tk, true);
+		timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
 		clock_was_set_delayed();
 	}
 	write_seqcount_end(&timekeeper_seq);
Signature · 1 line
1.8.3.2
#2 lore raw

RFC PATCH 2/3 timekeeping: Fix missing timekeeping_update in suspend path

In reply to John Stultz

Since 48cdc135d4840 (Implement a shadow timekeeper), we have to call timekeeping_update() after any adjustment to the timekeeping structure in order to make sure that any adjustments to the structure persist.

In the timekeeping suspend path, we udpate the timekeeper structure, so we should be sure to update the shadow-timekeeper before releasing the timekeeping locks. Currently this isn't done.

In most cases, the next time related code to run would be timekeeping_resume, which does update the shadow-timekeeper, but in an abundence of caution, this patch adds the call to timekeeping_update() in the suspend path.

Cc Thomas Gleixner <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Richard Cochran <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc stable <[email protected]> #3.10+
---
 kernel/time/timekeeping.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/time/timekeeping.c +2 -0
index 6bad3d9..c615e9d 100644
@@ -1024,6 +1024,8 @@ static int timekeeping_suspend(void)
 		timekeeping_suspend_time =
 			timespec_add(timekeeping_suspend_time, delta_delta);
 	}
+
+	timekeeping_update(tk, TK_MIRROR);
 	write_seqcount_end(&timekeeper_seq);
 	raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
 
Signature · 1 line
1.8.3.2
#3 lore raw

RFC PATCH 3/3 timekeeping: Fix potential lost pv notification of time change

In reply to John Stultz

In 780427f0e11 (Indicate that clock was set in the pvclock gtod notifier), logic was added to pass a CLOCK_WAS_SET notification to the pvclock notifier chain.

While that patch added a action flag returned from accumulate_nsecs_to_secs(), it only uses the returned value in one location, and not in the logarithmic accumulation.

This means if a leap second triggered during the logarithmic accumulation (which is most likely where it would happen), the notification that the clock was set would not make it to the pv notifiers.

This patch extends the logarithmic_accumulation pass down that action flag so proper notification will occur.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc David Vrabel <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Richard Cochran <[email protected]>
Cc stable <[email protected]> #3.11+
---
 kernel/time/timekeeping.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/time/timekeeping.c +5 -5
index c615e9d..e429229 100644
@@ -1297,7 +1297,7 @@ static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk)
  * Returns the unconsumed cycles.
  */
 static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,
-						u32 shift)
+						u32 shift, int *action)
 {
 	cycle_t interval = tk->cycle_interval << shift;
 	u64 raw_nsecs;
@@ -1311,7 +1311,7 @@ static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,
 	tk->cycle_last += interval;
 
 	tk->xtime_nsec += tk->xtime_interval << shift;
-	accumulate_nsecs_to_secs(tk);
+	*action |= accumulate_nsecs_to_secs(tk);
 
 	/* Accumulate raw time */
 	raw_nsecs = (u64)tk->raw_interval << shift;
@@ -1369,7 +1369,7 @@ static void update_wall_time(void)
 	struct timekeeper *tk = &shadow_timekeeper;
 	cycle_t offset;
 	int shift = 0, maxshift;
-	unsigned int action;
+	unsigned int action = 0;
 	unsigned long flags;
 
 	raw_spin_lock_irqsave(&timekeeper_lock, flags);
@@ -1404,7 +1404,7 @@ static void update_wall_time(void)
 	maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
 	shift = min(shift, maxshift);
 	while (offset >= tk->cycle_interval) {
-		offset = logarithmic_accumulation(tk, offset, shift);
+		offset = logarithmic_accumulation(tk, offset, shift, &action);
 		if (offset < tk->cycle_interval<<shift)
 			shift--;
 	}
@@ -1422,7 +1422,7 @@ static void update_wall_time(void)
 	 * Finally, make sure that after the rounding
 	 * xtime_nsec isn't larger than NSEC_PER_SEC
 	 */
-	action = accumulate_nsecs_to_secs(tk);
+	action |= accumulate_nsecs_to_secs(tk);
 
 	write_seqcount_begin(&timekeeper_seq);
 	/* Update clock->cycle_last with the new value */
Signature · 1 line
1.8.3.2
#4 lore raw

Re: [Xen-devel] [RFC][PATCH 3/3] timekeeping: Fix potential lost pv notification of time change

kernel/time/timekeeping.c static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk) · line 1300

Quoted 4 lines
Quoted 1 line
Quoted 1 line · John Stultz

static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,

kernel/time/timekeeping.c +1 -1
-						u32 shift)
+						u32 shift, int *action)

With plain int used here, ...

kernel/time/timekeeping.c static void update_wall_time(void) · line 1372

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1369,7 +1369,7 @@ static void update_wall_time(void)
 	struct timekeeper *tk = &shadow_timekeeper;
 	cycle_t offset;
 	int shift = 0, maxshift;
-	unsigned int action;
+	unsigned int action = 0;

... and unsigned int used here, ...

kernel/time/timekeeping.c static void update_wall_time(void) · line 1407

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1404,7 +1404,7 @@ static void update_wall_time(void)
 	maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
 	shift = min(shift, maxshift);
 	while (offset >= tk->cycle_interval) {
-		offset = logarithmic_accumulation(tk, offset, shift);
+		offset = logarithmic_accumulation(tk, offset, shift, &action);

... does this compile without warning for you?

Jan

#5 lore raw

Re: [Xen-devel] [RFC][PATCH 3/3] timekeeping: Fix potential lost pv notification of time change

Quoted 14 lines · Jan Beulich

kernel/time/timekeeping.c static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk) · line 1300

Quoted 4 lines
Quoted 1 line
Quoted 1 line · John Stultz

static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,

kernel/time/timekeeping.c +1 -1
-						u32 shift)
+						u32 shift, int *action)

With plain int used here, ...

kernel/time/timekeeping.c static void update_wall_time(void) · line 1372

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1369,7 +1369,7 @@ static void update_wall_time(void)
 	struct timekeeper *tk = &shadow_timekeeper;
 	cycle_t offset;
 	int shift = 0, maxshift;
-	unsigned int action;
+	unsigned int action = 0;

... and unsigned int used here, ...

Quoted 7 lines

kernel/time/timekeeping.c static void update_wall_time(void) · line 1407

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1404,7 +1404,7 @@ static void update_wall_time(void)
 	maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
 	shift = min(shift, maxshift);
 	while (offset >= tk->cycle_interval) {
-		offset = logarithmic_accumulation(tk, offset, shift);
+		offset = logarithmic_accumulation(tk, offset, shift, &action);

... does this compile without warning for you?

It does compile without a warning, but I'll fix it none the less!

Thanks for pointing this out!
-john

#6 lore raw

Re: [RFC][PATCH 3/3] timekeeping: Fix potential lost pv notification of time change

In reply to Jan Beulich
Quoted 14 lines · Jan Beulich

kernel/time/timekeeping.c static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk) · line 1300

Quoted 4 lines
Quoted 1 line
Quoted 1 line · John Stultz

static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,

kernel/time/timekeeping.c +1 -1
-						u32 shift)
+						u32 shift, int *action)

With plain int used here, ...

kernel/time/timekeeping.c static void update_wall_time(void) · line 1372

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1369,7 +1369,7 @@ static void update_wall_time(void)
 	struct timekeeper *tk = &shadow_timekeeper;
 	cycle_t offset;
 	int shift = 0, maxshift;
-	unsigned int action;
+	unsigned int action = 0;

... and unsigned int used here, ...

Quoted 7 lines

kernel/time/timekeeping.c static void update_wall_time(void) · line 1407

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1404,7 +1404,7 @@ static void update_wall_time(void)
 	maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
 	shift = min(shift, maxshift);
 	while (offset >= tk->cycle_interval) {
-		offset = logarithmic_accumulation(tk, offset, shift);
+		offset = logarithmic_accumulation(tk, offset, shift, &action);

... does this compile without warning for you?

It does compile without a warning, but I'll fix it none the less!

Thanks for pointing this out!
-john

#7 lore raw

Re: [RFC][PATCH 3/3] timekeeping: Fix potential lost pv notification of time change

In reply to John Stultz

kernel/time/timekeeping.c static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk) · line 1300

Quoted 4 lines
Quoted 1 line
Quoted 1 line · John Stultz

static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,

kernel/time/timekeeping.c +1 -1
-						u32 shift)
+						u32 shift, int *action)

With plain int used here, ...

kernel/time/timekeeping.c static void update_wall_time(void) · line 1372

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1369,7 +1369,7 @@ static void update_wall_time(void)
 	struct timekeeper *tk = &shadow_timekeeper;
 	cycle_t offset;
 	int shift = 0, maxshift;
-	unsigned int action;
+	unsigned int action = 0;

... and unsigned int used here, ...

kernel/time/timekeeping.c static void update_wall_time(void) · line 1407

Quoted 6 lines
kernel/time/timekeeping.c +1 -1
@@ -1404,7 +1404,7 @@ static void update_wall_time(void)
 	maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
 	shift = min(shift, maxshift);
 	while (offset >= tk->cycle_interval) {
-		offset = logarithmic_accumulation(tk, offset, shift);
+		offset = logarithmic_accumulation(tk, offset, shift, &action);

... does this compile without warning for you?

Jan

#8 lore raw

RFC PATCH 3/3 timekeeping: Fix potential lost pv notification of time change

In reply to John Stultz

In 780427f0e11 (Indicate that clock was set in the pvclock gtod notifier), logic was added to pass a CLOCK_WAS_SET notification to the pvclock notifier chain.

While that patch added a action flag returned from accumulate_nsecs_to_secs(), it only uses the returned value in one location, and not in the logarithmic accumulation.

This means if a leap second triggered during the logarithmic accumulation (which is most likely where it would happen), the notification that the clock was set would not make it to the pv notifiers.

This patch extends the logarithmic_accumulation pass down that action flag so proper notification will occur.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc David Vrabel <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc Prarit Bhargava <[email protected]>
Cc Richard Cochran <[email protected]>
Cc stable <[email protected]> #3.11+
---
 kernel/time/timekeeping.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/time/timekeeping.c +5 -5
index c615e9d..e429229 100644
@@ -1297,7 +1297,7 @@ static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk)
  * Returns the unconsumed cycles.
  */
 static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,
-						u32 shift)
+						u32 shift, int *action)
 {
 	cycle_t interval = tk->cycle_interval << shift;
 	u64 raw_nsecs;
@@ -1311,7 +1311,7 @@ static cycle_t logarithmic_accumulation(struct timekeeper *tk, cycle_t offset,
 	tk->cycle_last += interval;
 
 	tk->xtime_nsec += tk->xtime_interval << shift;
-	accumulate_nsecs_to_secs(tk);
+	*action |= accumulate_nsecs_to_secs(tk);
 
 	/* Accumulate raw time */
 	raw_nsecs = (u64)tk->raw_interval << shift;
@@ -1369,7 +1369,7 @@ static void update_wall_time(void)
 	struct timekeeper *tk = &shadow_timekeeper;
 	cycle_t offset;
 	int shift = 0, maxshift;
-	unsigned int action;
+	unsigned int action = 0;
 	unsigned long flags;
 
 	raw_spin_lock_irqsave(&timekeeper_lock, flags);
@@ -1404,7 +1404,7 @@ static void update_wall_time(void)
 	maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1;
 	shift = min(shift, maxshift);
 	while (offset >= tk->cycle_interval) {
-		offset = logarithmic_accumulation(tk, offset, shift);
+		offset = logarithmic_accumulation(tk, offset, shift, &action);
 		if (offset < tk->cycle_interval<<shift)
 			shift--;
 	}
@@ -1422,7 +1422,7 @@ static void update_wall_time(void)
 	 * Finally, make sure that after the rounding
 	 * xtime_nsec isn't larger than NSEC_PER_SEC
 	 */
-	action = accumulate_nsecs_to_secs(tk);
+	action |= accumulate_nsecs_to_secs(tk);
 
 	write_seqcount_begin(&timekeeper_seq);
 	/* Update clock->cycle_last with the new value */
Signature · 1 line
1.8.3.2