Skip to contentSkip to conversation

nanosleep() granularity bumps up in 2.5.64 (was: [PATCH] settimeofday() not synchronised with gettimeofday())

BUG In review Eric Piel 6 participants 9 messages Mar 14, 2003 – Mar 24, 2003 2 messages not in the archive Timekeeping, clocksource core, NTP, alarmtimer

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

Eric Piel

author
#2 lore raw
Quoted 1 line · David Mosberger

Are you running ntp?

Yes (I hadn't noticed it) but it was not connected to any server and disabling it doesn't change the results.

Quoted 13 lines

On 2.5:

        $ time sleep 16
        real    0m16.002s
        user    0m0.001s
        sys     0m0.002s
        $ time sleep 16.02
        real    0m25.189s
        user    0m0.000s
        sys     0m0.001s
Quoted 4 lines

So clearly something very strange is going on. My suspicion is that the bug was introduced back when x86 switched from 100Hz to 1000Hz ticks, but that's just a guess. Eric, would you be able/willing/interested to look into this?

Sure, I aim at porting the high resolution timers but any annoying bug related to the time can be interesting to remove.

Coincidently Vita has just reported a bug on the lkml which, after a closer look, seems to be the same:

Quoted 5 lines

When playing with select() timeout values I found that granularity
of nanosleep() in 2.5.64 kernel bumps to 256 msec. Trying to get finer granularity it ends up sleeping to the next multiple of 256 msec

From what I understand their is a bug in the timers that causes a big

granularity. The case of Vita is a very good example. Also, after 16s it seems the granuality (slowly?!) jumps from 1/64th s to 16s! :
sleep requested time obtained

14.000000000     14.006201744
15.000000000     15.006647110
16.000000000     16.007089615
17.000000000     18.742679596
18.000000000     32.014190674
19.000000000     32.014190674
20.000000000     32.014190674

I think lines like that from patch-2.5.64 are very suspicious to be related to the bug:
+ base->timer_jiffies = INITIAL_JIFFIES;
+ base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
+ base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
+ base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
+ base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) &
TVN_MASK;
+ base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) &
TVN_MASK;

Any idea/sugestion/patch is welcomed. Whatever, I will try to fix this as soon as I'm back from my week end :-)

Eric

#3 lore raw
Quoted 10 lines · Eric Piel

I think lines like that from patch-2.5.64 are very suspicious to be related to the bug:

(quoted diff) +5 -0
+	base->timer_jiffies = INITIAL_JIFFIES;
+	base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
+	base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
+	base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
+	base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) &

TVN_MASK;

(quoted diff) +1 -0
+	base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) &

TVN_MASK;

No, I don't think so. Those lines are for starting `jiffies' at a very high number so we spot jiffie-wrap bugs early on.

Signature · 3 lines
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk
#4 lore raw
Quoted 14 lines · Matthew Wilcox
Quoted 10 lines · Eric Piel

I think lines like that from patch-2.5.64 are very suspicious to be related to the bug:

(quoted diff) +5 -0
+	base->timer_jiffies = INITIAL_JIFFIES;
+	base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
+	base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
+	base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
+	base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) &

TVN_MASK;

(quoted diff) +1 -0
+	base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) &

TVN_MASK;

No, I don't think so. Those lines are for starting `jiffies' at a very high number so we spot jiffie-wrap bugs early on.

The nanosleep() bug narrowed down to 2.5.63-bk2. That's version, the "initial
jiffies" patch went in. And yes, it's on i686 machine.

Cheers,
Vita

#5 lore raw

Re: [BUG] nanosleep() granularity bumps up in 2.5.64

Quoted 16 lines · Vitezslav Samel
Quoted 14 lines · Matthew Wilcox
Quoted 10 lines · Eric Piel

I think lines like that from patch-2.5.64 are very suspicious to be related to the bug:

(quoted diff) +5 -0
+	base->timer_jiffies = INITIAL_JIFFIES;
+	base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
+	base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
+	base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
+	base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) &

TVN_MASK;

(quoted diff) +1 -0
+	base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) &

TVN_MASK;

No, I don't think so. Those lines are for starting `jiffies' at a very high number so we spot jiffie-wrap bugs early on.

Quoted 2 lines

The nanosleep() bug narrowed down to 2.5.63-bk2. That's version, the "initial
jiffies" patch went in. And yes, it's on i686 machine.

You can easily check whether it's connected with this change by setting INITIAL_JIFFIES to zero. This should exactly recover the previous situation.
I.e., something like the following (untested, hand-crafted) patch:

include/linux/time.h +1 -1
@@ -28,7 +28,7 @@
  * Have the 32 bit jiffies value wrap 5 minutes after boot
  * so jiffies wrap bugs show up earlier.
  */
- #define INITIAL_JIFFIES ((unsigned int) (-300*HZ))
+ #define INITIAL_JIFFIES 0
 
 /*
  * Change timeval to jiffies, trying to avoid the

Tim

#6 lore raw

[BUG & WORKAROUND] nanosleep() granularity bumps up in 2.5.64

Quoted 8 lines · Tim Schmielau
Quoted 2 lines · Vitezslav Samel

The nanosleep() bug narrowed down to 2.5.63-bk2. That's version, the "initial
jiffies" patch went in. And yes, it's on i686 machine.

You can easily check whether it's connected with this change by setting INITIAL_JIFFIES to zero. This should exactly recover the previous situation.

OK. I've done the test myself and I plead guilty. As a temporary workaround you can apply the following patch:

include/linux/time.h renamed from include/linux/time.h.orig +1 -1
@@ -31,7 +31,7 @@
  * Have the 32 bit jiffies value wrap 5 minutes after boot
  * so jiffies wrap bugs show up earlier.
  */
-#define INITIAL_JIFFIES ((unsigned int) (-300*HZ))
+#define INITIAL_JIFFIES 0
 
 /*
  * Change timeval to jiffies, trying to avoid the

Still, after half an hour of glancing at the code I can't see my mistake. I've re-checked that the problem does not occur with the original "initial jiffies" patch for 2.4. So I must have missed a (subtle?) difference between 2.4 and 2.5 when I did the forward-port.

Sorry,
Tim

#7 lore raw

PATCH 1/1 fix nanosleep() granularity bumps

Quoted 2 lines · Tim Schmielau

I've re-checked that the problem does not occur with the original "initial jiffies" patch for 2.4.

Stupid me - 2.4 with the patch has the same problem, it just takes 10x as long to show up, a range that was not covered by the testcase.

The actual problem is that I didn't fully understand the, well, 'elaborated' way of counting jiffies in the timer cascade:

When starting out with timer_jiffies=0, the timer cascade is (unneccessarily) triggered on the first timer interrupt, incrementing all the higher indices.
When starting with any other initial jiffies value, we miss that and end up with all higher indices being off by one.

Fix is as follows:

kernel/timer.c +8 -4
@@ -1183,10 +1183,14 @@
 
 	base->timer_jiffies = INITIAL_JIFFIES;
 	base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
-	base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
-	base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
-	base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) & TVN_MASK;
-	base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) & TVN_MASK;
+	base->tv2.index = ((INITIAL_JIFFIES >> TVR_BITS)
+	                   + (INITIAL_JIFFIES!=0)) & TVN_MASK;
+	base->tv3.index = ((INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS))
+	                   + (INITIAL_JIFFIES!=0)) & TVN_MASK;
+	base->tv4.index = ((INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS))
+	                   + (INITIAL_JIFFIES!=0)) & TVN_MASK;
+	base->tv5.index = ((INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS))
+	                   + (INITIAL_JIFFIES!=0)) & TVN_MASK;
 }
 
 static int __devinit timer_cpu_notify(struct notifier_block *self,

Sorry for all the trouble,
Tim

#8 lore raw

Re: [PATCH] fix nanosleep() granularity bumps

Quoted 9 lines · Tim Schmielau
Quoted 2 lines · Tim Schmielau

I've re-checked that the problem does not occur with the original "initial jiffies" patch for 2.4.

Stupid me - 2.4 with the patch has the same problem, it just takes 10x as long to show up, a range that was not covered by the testcase.

Quoted 8 lines

The actual problem is that I didn't fully understand the, well, 'elaborated' way of counting jiffies in the timer cascade:

When starting out with timer_jiffies=0, the timer cascade is (unneccessarily) triggered on the first timer interrupt, incrementing all the higher indices.
When starting with any other initial jiffies value, we miss that and end up with all higher indices being off by one.

Suggest the attached patch as a fix instead - easier to understand I think and works for every possible start value. This is what I made for Andrea Arcangeli many years ago...

kernel/timer.c +8 -5
diff -ur linux-2.5.65/kernel/timer.c linux-2.5.65-new/kernel/timer.c
@@ -1182,11 +1182,14 @@
 		INIT_LIST_HEAD(base->tv1.vec + j);
 
 	base->timer_jiffies = INITIAL_JIFFIES;
-	base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
-	base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
-	base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
-	base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) & TVN_MASK;
-	base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) & TVN_MASK;
+	base->tv1.index = (1 + (INITIAL_JIFFIES - 1)) & TVR_MASK;
+	base->tv2.index = (1 + ((INITIAL_JIFFIES - 1) >> TVR_BITS)) & TVN_MASK;
+	base->tv3.index = (1 + ((INITIAL_JIFFIES - 1)
+				>> (TVR_BITS + TVN_BITS))) & TVN_MASK;
+	base->tv4.index = (1 + ((INITIAL_JIFFIES - 1)
+				>> (TVR_BITS + 2 * TVN_BITS))) & TVN_MASK;
+	base->tv5.index = (1 + ((INITIAL_JIFFIES - 1)
+				>> (TVR_BITS + 3 * TVN_BITS))) & TVN_MASK;
 }
 	
 static int __devinit timer_cpu_notify(struct notifier_block *self, 

- Finn Arne

#9 lore raw

Re: [PATCH] fix nanosleep() granularity bumps

Quoted 1 line · Tim Schmielau

[...]

Quoted 17 lines

Suggest the attached patch as a fix instead - easier to understand I think and works for every possible start value. This is what I made for Andrea Arcangeli many years ago...

diff -ur linux-2.5.65/kernel/timer.c linux-2.5.65-new/kernel/timer.c

kernel/timer.c +1 -5
@@ -1182,11 +1182,14 @@
 		INIT_LIST_HEAD(base->tv1.vec + j);
 
 	base->timer_jiffies = INITIAL_JIFFIES;
-	base->tv1.index = INITIAL_JIFFIES & TVR_MASK;
-	base->tv2.index = (INITIAL_JIFFIES >> TVR_BITS) & TVN_MASK;
-	base->tv3.index = (INITIAL_JIFFIES >> (TVR_BITS+TVN_BITS)) & TVN_MASK;
-	base->tv4.index = (INITIAL_JIFFIES >> (TVR_BITS+2*TVN_BITS)) & TVN_MASK;
-	base->tv5.index = (INITIAL_JIFFIES >> (TVR_BITS+3*TVN_BITS)) & TVN_MASK;
+	base->tv1.index = (1 + (INITIAL_JIFFIES - 1)) & TVR_MASK;

kernel/timer.c line 1195

Quoted 10 lines
kernel/timer.c +7 -0
+	base->tv2.index = (1 + ((INITIAL_JIFFIES - 1) >> TVR_BITS)) & TVN_MASK;
+	base->tv3.index = (1 + ((INITIAL_JIFFIES - 1)
+				>> (TVR_BITS + TVN_BITS))) & TVN_MASK;
+	base->tv4.index = (1 + ((INITIAL_JIFFIES - 1)
+				>> (TVR_BITS + 2 * TVN_BITS))) & TVN_MASK;
+	base->tv5.index = (1 + ((INITIAL_JIFFIES - 1)
+				>> (TVR_BITS + 3 * TVN_BITS))) & TVN_MASK;
 }
 
 static int __devinit timer_cpu_notify(struct notifier_block *self,

Too late - the whole tv[1-5].index duplication is gone already after a cleanup done by George Anzinger.

Tim

#10 lore raw
In reply to Eric Piel
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Eric Piel

Eric> Sure, I aim at porting the high resolution timers but any Eric> annoying bug related to the time can be interesting to remove.

Great! It would be great to have someone who can focus on that.

Eric> Any idea/sugestion/patch is welcomed. Whatever, I will try to Eric> fix this as soon as I'm back from my week end :-)

Does the x86 show the same behavior? That would be a useful start to narrow down the problem.

	--david